Skip to content

Count streamed usage once when a backend repeats the running total - #86

Merged
linted merged 1 commit into
mainfrom
worktree-opencode-streamed-usage
Nov 18, 2026
Merged

linted merged 1 commit into
mainfrom
worktree-opencode-streamed-usage

Conversation

@linted

@linted linted commented Nov 18, 2026

Copy link
Copy Markdown
Owner

Problem

Some backends behind the OpenCode gateway attach the cumulative usage to every
streamed chunk rather than once at the end. LangChain sums chunk usage when merging,
so a 34s GLM-5.3 call was reported (TUI + LangSmith) as 53.3M input / 1.38M output
tokens — ~405 chunks each repeating a ~131k prompt (another call: 394,704 = 3 × 131,568).
OpenCode billed the whole day at 11.6M input / 242.5k output; LangSmith showed 65.2M / 1.59M.

Fix

_ChatDeepSeekFixed._stream/_astream rewrite each chunk's usage as the delta from
the previous running total. That sums correctly whether a backend reports usage once
at the end or on every chunk.

Testing

lint, typecheck, ./scripts/test.sh pass (312). New tests cover a cumulative-per-chunk
stream and a normal single final usage chunk. Live-checked normal streaming through
build_model; the misbehaving backend couldn't be reproduced on demand.

Some backends behind the OpenCode gateway attach the cumulative usage to
every streamed chunk rather than once at the end. LangChain sums chunk
usage when merging, so a 34s GLM-5.3 call was reported (in the TUI and
LangSmith) as 53.3M input / 1.38M output tokens: ~405 chunks each
repeating a ~131k prompt. OpenCode billed the day at 11.6M input.

_ChatDeepSeekFixed now rewrites each chunk's usage as the delta from the
previous total, which sums correctly whether usage arrives once or on
every chunk.
@linted
linted merged commit ae5f345 into main Nov 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant