Repository navigation
Count streamed usage once when a backend repeats the running total - #86
Merged
Merged
Conversation
Some backends behind the OpenCode gateway attach the cumulative usage to every streamed chunk rather than once at the end. LangChain sums chunk usage when merging, so a 34s GLM-5.3 call was reported (in the TUI and LangSmith) as 53.3M input / 1.38M output tokens: ~405 chunks each repeating a ~131k prompt. OpenCode billed the day at 11.6M input. _ChatDeepSeekFixed now rewrites each chunk's usage as the delta from the previous total, which sums correctly whether usage arrives once or on every chunk.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Some backends behind the OpenCode gateway attach the cumulative usage to every
streamed chunk rather than once at the end. LangChain sums chunk usage when merging,
so a 34s GLM-5.3 call was reported (TUI + LangSmith) as 53.3M input / 1.38M output
tokens — ~405 chunks each repeating a ~131k prompt (another call: 394,704 = 3 × 131,568).
OpenCode billed the whole day at 11.6M input / 242.5k output; LangSmith showed 65.2M / 1.59M.
Fix
_ChatDeepSeekFixed._stream/_astreamrewrite each chunk's usage as the delta fromthe previous running total. That sums correctly whether a backend reports usage once
at the end or on every chunk.
Testing
lint, typecheck,
./scripts/test.shpass (312). New tests cover a cumulative-per-chunkstream and a normal single final usage chunk. Live-checked normal streaming through
build_model; the misbehaving backend couldn't be reproduced on demand.