fix(gateway): meter OpenAI usage from the shapes the wire actually sends - #262
Open
ophiocus wants to merge 2 commits into
Open
fix(gateway): meter OpenAI usage from the shapes the wire actually sends#262ophiocus wants to merge 2 commits into
ophiocus wants to merge 2 commits into
Conversation
The gateway parsed OpenAI usage with Responses-only, top-level names, so Chat Completions (prompt_tokens/completion_tokens) and streamed Responses (usage nested in the response.completed envelope - the wire Codex speaks) both metered to zero: hard budgets never accumulated for Codex runs. The stubs encoded the same wrong shapes, so the assertions could not catch it (theam#232). Both stubs now emit the wire truth - with those stubs alone, tests 3 and 3b fail against the old parser at "expected +0 to be 600000/750000". usageFromJson accepts both naming schemes, the stream path also reads the response envelope, and cached tokens are subtracted from input: OpenAI reports input INCLUSIVE of cached while costCents sums buckets additively in the Anthropic convention, so the old accounting billed cached tokens at both the input and cache-read rates. Metering assertions now pin the full bucket math (600k/400k cached at 216 cents; 750k/250k/200k at 987.5). Closes theam#232 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #232 — and full credit to @Julian-Genuario, whose diagnosis was exact down to the sentence "the tests encode the wrong shapes." I claimed this in the issue thread with right of way offered; the courtesy window has passed, so here is the fix, built the way #181 was.
Stubs first, so the tests bite (the #181 pattern)
Both OpenAI stubs now emit what the wire actually sends — Chat Completions speaks
prompt_tokens/completion_tokens(withprompt_tokens_details.cached_tokenson the final include_usage chunk), and the/responsesstub streams the real envelope: usage nested inside{"type":"response.completed","response":{...}}, never at the frame's top level. With the corrected stubs alone, tests 3 and 3b fail against the old parser exactly as the issue predicted:Then the parser
usageFromJsonaccepts both naming schemes; the stream path also reads theresponse.completedenvelope (the wire Codex speaks) alongside top-level usage.costCentssums buckets additively in the Anthropic convention — so cached tokens are subtracted from input, or they get billed at both the input and cache-read rates.Pinned money math
The assertions now check the full buckets, not just presence: 1M prompt with 400k cached + 1M completion on gpt-5.5-mini = 216¢ (was silently 225 with cached double-billed and names lucky-matched); the streamed Codex path (1M/250k cached/200k out on gpt-5.6-sol) = 987.5¢ — previously zero, which is the hard-budget bypass in the issue title.
Gateway suite: 57 tests, 0 failures, tsc clean. Adjacent, not overlapping: #248/#251/#255 handle incomplete usage reconciliation; this handles usage that arrives complete but was unparseable.