fix(llm): propagate token usage through review/scan/stream paths#323
Merged
Merged
Conversation
Three bugs caused tokens_used to always be None or partial: 1. parse_review_response/parse_scan_response hardcoded Ok((..., None)), discarding the Usage object from the API response. Fixed by threading Option<&Usage> through both functions and converting via usage_to_token_usage(). 2. chat_completion_stream only collected delta.content, ignoring the usage field in the final SSE chunk. Now sends stream_options.include_usage and parses usage from top-level or choices[0].delta.usage. 3. scan multi-batch total_tokens was overwritten per batch instead of accumulated. Fixed with proper accumulation of input/output/cost. Also: - Extracted magic numbers (20, 60_000) to named constants - Usage struct now accepts camelCase aliases (promptTokens etc) - chat_completion returns (String, Option<Usage>) tuple - Added 4 regression tests for token usage threading - Bumped version 0.6.1 → 0.6.2 All 590 tests pass (568 unit + 16 CLI + 6 config).
ajianaz
force-pushed
the
fix/token-usage-tracking-and-hardcode-cleanup
branch
from
9a29e6f to
24999b9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes 3 bugs where
tokens_usedwas alwaysNoneor partial, plus hardcode cleanup.Bugs Fixed
tokens_usedalwaysNone— parse functions hardcodedOk((..., None)), discarding APIUsagecora review --streamignoredusagefield in SSE chunkscora scanmulti-batch overwrote (not accumulated) token countsChanges
chat_completion()returns(String, Option<Usage>)tuplechat_completion_stream()sendsstream_options: {include_usage: true}+ parses usage from final SSE chunkparse_review_response/parse_scan_responseacceptOption<&Usage>and convert viausage_to_token_usage()20,60_000) extracted to named constantsUsagestruct accepts camelCase aliases + handles total-only providersTest Plan
cargo clippy --all-targets— zero warningsCloses #322