Skip to content

Fix token cost calculation discrepancies#995

Merged
robinebers merged 3 commits into
mainfrom
fix/token-cost-calculation-disperancies
Jul 14, 2026
Merged

Fix token cost calculation discrepancies#995
robinebers merged 3 commits into
mainfrom
fix/token-cost-calculation-disperancies

Conversation

@validatedev

@validatedev validatedev commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

TL;DR

Corrects local token-cost estimates for large and priority-tier Codex requests, preserves whether cache discounts were actually published, and counts nested Claude advisor usage under the advisor model. It also normalizes Codex -fast aliases before provider pricing and verifies that persisted claude -p usage is included.

What was happening

  • Codex requests above 272K input tokens were always priced at base rates, even when OpenAI applies higher rates to the whole request.
  • Codex priority pricing fell back to one generic multiplier, which underpriced GPT-5.5 and could apply Cursor-specific fast multipliers to GPT-5.6.
  • Codex model aliases ending in -fast could retain Cursor-scaled rates and bypass the Codex long-context and priority switches.
  • Pricing catalogs synthesized a 10% cache-read rate without retaining whether the provider published that discount; models with no cache discount could therefore be underpriced.
  • Claude advisor iterations were ignored, leaving their tokens and locally imputed costs out of the spend totals.
  • Persisted Claude print-mode usage was not explicitly covered or documented, making claude -p and --no-session-persistence behavior unclear.

What this changes

  • Adds a configurable long-context threshold to the shared pricing engine and applies Codex's 272K threshold with model-specific GPT-5.4, GPT-5.5, and GPT-5.6 rates.
  • Applies provider-specific Codex priority multipliers: 2.5x for GPT-5.5 and 2x for supported GPT-5.4/GPT-5.6 models.
  • Resolves Codex -fast aliases through the unscaled base model, then applies the Codex multiplier exactly once for both short and long-context requests.
  • Carries cache-read provenance through live feeds, the supplement, compact caches, and the snapshot generator; Codex uses full input rates when no discount is published.
  • Keeps explicit no-discount handling for GPT-5.4 Pro and GPT-5.5 Pro compatible with older bundled snapshots.
  • Expands Claude advisor_message iterations into deduplicated entries priced under their own model while leaving ordinary iterations in the parent totals.
  • Adds regression coverage for persisted claude -p (sdk-cli) records and documents that --no-session-persistence writes no log for OpenUsage to read.
  • Documents the updated Codex and Claude spend behavior.

Heads-up

  • Claude log-reported usage.speed remains authoritative. No date-derived fast-mode override was added because post-retirement Opus 4.6 responses already report standard; overriding that field could misprice real events.
  • claude -p counts when Claude persists the session. Calls using --no-session-persistence cannot be recovered from Claude's local history because that flag intentionally prevents the usage record from being written.
  • Bundled pricing snapshots were not refreshed, avoiding unrelated catalog churn. The updated generator will persist cache-rate provenance on the next scheduled snapshot refresh, while current live refreshes persist it immediately.

Tests

  • swift test --filter 'CodexLogUsageScannerTests|ClaudeLogUsageScannerTests|ModelPricingTests'
  • swift test — 1,144 XCTest cases passed, 3 opt-in tests skipped; 3 Swift Testing cases passed.
  • ./script/build_and_run.sh verify — release build, signing, relaunch, and process verification succeeded. The existing icon compiler fallback used assets/AppIcon.prebuilt; the local build had no matching iCloud provisioning profile.

@robinebers robinebers added the gate-passed Passed the automated PR gatekeeper checks (gatekeeper skips it on later runs) label Jul 13, 2026 — with Cursor
@validatedev

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bfb325c802

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/OpenUsage/Providers/Codex/CodexLogUsageScanner.swift
validatedev and others added 2 commits July 13, 2026 20:32
…toml

The fast/priority flag was read from the current config.toml and applied
to the entire 30-day history, so enabling priority for a single task
retroactively repriced every past day (~2x) and published the inflated
figures to the synced iCloud history until the setting was reverted.

Codex CLI (since ~Jul 2026) records the tier in each session rollout as
thread_settings_applied lines. Track it per session the same way the
scanner already tracks the model from turn_context, stamp each event,
and price only the turns that actually ran fast at fast rates. Sessions
without tier metadata price at standard rates; config.toml is no longer
consulted at all.

Co-authored-by: Cursor <cursoragent@cursor.com>
@robinebers

Copy link
Copy Markdown
Owner

Pushed cfdfdd0, which closes the remaining gap in how the fast/priority tier is detected.

Why

The multiplier corrections in this PR are right (verified against OpenAI's published priority table: 2.5× for GPT-5.5, 2× for GPT-5.4/5.6), but the detection still read service_tier from the current config.toml and applied it to the entire 30-day history. That caused a real incident: priority was enabled for one task, and every past Codex day repriced ~2× — confirmed by comparing iCloud history snapshots of the same token days ($0.77/MTok vs $1.55/MTok, an exact 2.0×). The inflated history then synced to every Mac until the config was flipped back.

What changed

  • The scanner now tracks the tier per session from thread_settings_applied lines (Codex CLI records these since ~Jul 2026), exactly like it already tracks the model from turn_context. Each event is stamped isFast and only turns that actually ran fast/priority price at fast rates — including mid-session tier switches.
  • config.toml is no longer consulted at all. usesFastServiceTier is deleted and the global fastTier parameter is gone from aggregate.
  • Sessions with no recorded tier (pre-Jul-2026 logs, older CLIs) price at standard rates. Deliberate trade-off: the worst case becomes slightly under-pricing an old fast turn, instead of retroactively inflating 30 days of history and the synced iCloud documents.
  • Existing inflated history self-heals: the next refresh reprices from the logs and overwrites the device's iCloud history file.

Tests

  • New parse tests: tier tracked across thread_settings_applied transitions (defaultprioritydefault), fast spelling, and no-metadata sessions defaulting to standard.
  • New end-to-end regression: a home whose config.toml says service_tier = "priority" still prices at standard rates.
  • Full suite: 1,144 tests pass. Parity harness against real local logs matches the expected standard-rate figures.

@robinebers robinebers merged commit a0569bd into main Jul 14, 2026
2 checks passed
@robinebers robinebers deleted the fix/token-cost-calculation-disperancies branch July 14, 2026 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs gate-passed Passed the automated PR gatekeeper checks (gatekeeper skips it on later runs) provider tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants