Summary
With DeepSeek-V4-Flash-0731 (GGUF) served by llama-server with --jinja and tools armed, generation terminated mid-tag with finish_reason: "stop" on three consecutive agent turns — each time exactly at the point where the reasoning stream contained literal DSML closing fragments (</｜DSML｜parameter>, </｜DSML｜invoke>). The final streamed token was tool_c — an incomplete </｜DSML｜tool_calls> closing tag. The client-visible result is an agent that appears to stop mid-tool-call, and (because the truncated message is persisted into history) a self-perpetuating loop: each follow-up turn imitates the truncated pattern and gets cut at the same place.
Environment
llama-server build 10573, commit d775b8967 (master, 2026-08-22), AppleClang 21, Darwin arm64- Backends: Metal (M5 Max) + one RPC worker (DGX Spark GB10),
--split-mode layer
- Model:
unsloth/DeepSeek-V4-Flash-0731-GGUF UD-Q8_K_XL (5 shards) + dspark-DeepSeek-V4-Flash-0731-Q8_0.gguf draft
- Flags:
--spec-type draft-dspark --spec-draft-n-max 3 -c 393216 --flash-attn on --cache-type-k f16 --cache-type-v f16 --jinja --temp 0.6 --top-p 0.95
- Effective slot state observed:
chat_format: "peg-native", reasoning_format: "deepseek", generation_prompt: "<｜Assistant｜><think>"
Observed (3 consecutive turns, harness session transcript)
| Turn |
input tokens |
output tokens |
finish |
stream ends at |
| 1 |
~165,000 |
~120 (est.) |
stop |
reasoning text ...a1b2c3d + </｜DSML｜parameter> |
| 2 |
168,991 |
126 |
stop |
...a1b2c3 + </｜DSML｜parameter>\n</｜DSML｜invoke>\n</｜DSML｜tool_c |
| 3 |
169,132 |
119 |
stop |
same fragment, same cut point |
Final reasoning deltas for turn 3 (verbatim, from the client event log):
... 'c', '3', '</｜DSML｜', 'parameter', '>', '\n', '</｜DSML｜', 'inv', 'oke', '>', '\n', '</｜DSML｜', 'tool', '_c'
The model was composing a file-edit tool call whose parameter value contained backtick-quoted hex strings (`a1b2c3d4e5f6`). The reasoning text shows the model interleaving protocol markup into the thinking channel; the stream then terminates mid-tag. No EOS content token is visible in the client stream — the stop appears to come from the serving side's handling of the DSML structure, not from a deliberate EOS the model wrote.
Expected
Reasoning/thinking content should be treated as opaque text: a stray or partial DSML closing tag inside reasoning must not end generation. If the model truly opens a tool call, the parser should wait for a well-formed close; a malformed close should yield a parse error or plain text — never a silent mid-tag stop.
Minimal repro attempt (negative result at short context)
A direct request asking the model to quote </｜DSML｜parameter> inside its reasoning, with a dummy tool armed, completed normally at short context (finish: stop, 271 completion tokens, tag quoted intact). So the failure is not triggered by the tag alone — the incidents happened at ~165–169K context depth in an agentic multi-tool flow (the harness session had ~100 prior tool calls). Possibly depth-dependent (parser state / attention degradation at depth making the model emit protocol markup mid-reasoning more likely, and/or the parser's early-stop engaging once it does).
Why it matters
The truncated message is persisted by clients as a normal completed turn (finish: stop), and the model then imitates its own truncated pattern on the next turn — the failure is self-reinforcing within a session. Three identical stops in a row required abandoning the session.
Related
Happy to provide the full client event log (JSONL), server logs, or test patches.
Summary
With
DeepSeek-V4-Flash-0731(GGUF) served byllama-serverwith--jinjaand tools armed, generation terminated mid-tag withfinish_reason: "stop"on three consecutive agent turns — each time exactly at the point where the reasoning stream contained literal DSML closing fragments (</｜DSML｜parameter>,</｜DSML｜invoke>). The final streamed token wastool_c— an incomplete</｜DSML｜tool_calls>closing tag. The client-visible result is an agent that appears to stop mid-tool-call, and (because the truncated message is persisted into history) a self-perpetuating loop: each follow-up turn imitates the truncated pattern and gets cut at the same place.Environment
llama-serverbuild 10573, commitd775b8967(master, 2026-08-22), AppleClang 21, Darwin arm64--split-mode layerunsloth/DeepSeek-V4-Flash-0731-GGUFUD-Q8_K_XL (5 shards) +dspark-DeepSeek-V4-Flash-0731-Q8_0.ggufdraft--spec-type draft-dspark --spec-draft-n-max 3 -c 393216 --flash-attn on --cache-type-k f16 --cache-type-v f16 --jinja --temp 0.6 --top-p 0.95chat_format: "peg-native",reasoning_format: "deepseek",generation_prompt: "<｜Assistant｜><think>"Observed (3 consecutive turns, harness session transcript)
...a1b2c3d+</｜DSML｜parameter>...a1b2c3+</｜DSML｜parameter>\n</｜DSML｜invoke>\n</｜DSML｜tool_cFinal reasoning deltas for turn 3 (verbatim, from the client event log):
The model was composing a file-edit tool call whose parameter value contained backtick-quoted hex strings (
`a1b2c3d4e5f6`). The reasoning text shows the model interleaving protocol markup into the thinking channel; the stream then terminates mid-tag. No EOS content token is visible in the client stream — the stop appears to come from the serving side's handling of the DSML structure, not from a deliberate EOS the model wrote.Expected
Reasoning/thinking content should be treated as opaque text: a stray or partial DSML closing tag inside reasoning must not end generation. If the model truly opens a tool call, the parser should wait for a well-formed close; a malformed close should yield a parse error or plain text — never a silent mid-tag
stop.Minimal repro attempt (negative result at short context)
A direct request asking the model to quote
</｜DSML｜parameter>inside its reasoning, with a dummy tool armed, completed normally at short context (finish: stop, 271 completion tokens, tag quoted intact). So the failure is not triggered by the tag alone — the incidents happened at ~165–169K context depth in an agentic multi-tool flow (the harness session had ~100 prior tool calls). Possibly depth-dependent (parser state / attention degradation at depth making the model emit protocol markup mid-reasoning more likely, and/or the parser's early-stop engaging once it does).Why it matters
The truncated message is persisted by clients as a normal completed turn (
finish: stop), and the model then imitates its own truncated pattern on the next turn — the failure is self-reinforcing within a session. Three identical stops in a row required abandoning the session.Related
<｜DSML｜tool_calls>START token (long context) vllm-project/vllm#48931 — DSML parser fragility when the model omits the<｜DSML｜tool_calls>START wrapper (V4, long context)Happy to provide the full client event log (JSONL), server logs, or test patches.