Find out why your prompt cache is not paying.
Every major provider discounts a repeated prompt prefix heavily. The discount applies only when the prefix is identical to the last one, and when it is not, nothing happens: no error, no warning, a normal response, full price. Groundhog finds the reasons that does not happen, names each one, and says what to do about it.
It runs locally. Nothing leaves your machine, there is no account, and the check path makes no network calls.
Status: early. Anthropic and OpenAI are covered, plus a generic OpenAI compatible endpoint. Most provider facts, including the model minimums, are drawn from documentation and marked unverified until a conformance run measures them against the live API. Every number the tool prints carries its confidence, so you always know what is measured and what is a guess.
- Install
- Five minutes to an answer
- What it tells you
- How it works
- What it checks today
- The core checks
- The audit
- Catching the deploy that cold starts the cache
- Measuring a provider instead of trusting its docs
- Three things it will not do
- Correcting the provider data
- Command reference
- Development
Requires Node 20 or newer.
Not yet published to npm, so run it straight from this repository. It builds on install, so no separate build step is needed:
npx github:swarina/groundhog doctor request.json --model claude-sonnet-4-5Or from a clone, which is also the way to use it as a library:
git clone https://github.com/swarina/groundhog
cd groundhog
npm install
node dist/cli/index.js doctor request.json --model claude-sonnet-4-5The examples below write groundhog for the command. Read that as
npx github:swarina/groundhog, or link the clone with npm link to get a real
groundhog on your path.
Groundhog looks at the request your application actually sends, which is not always the request your code appears to build. An SDK normalises content shapes, fills in defaults and serialises before anything reaches the provider, and the provider hashes what arrived. So the accurate place to measure is the wire.
Record what your existing tests already send:
// install first: npm install github:swarina/groundhog
import { capture } from 'groundhog/capture';
const recorder = capture();
test('support agent prompt is cacheable', async () => {
await runMyAgent({ query: 'where is my order' });
const [report] = recorder.inspect({ model: 'claude-sonnet-4-5' });
expect(report.ok).toBe(true);
});Nothing is sent. The recorder answers with a minimal valid response so the code under test carries on, which means no keys, no cost and no network. If it captures nothing it raises an error rather than reporting a pass, because a check that inspected no requests has not verified anything.
If you already have a request body saved, skip the recorder:
groundhog doctor request.json --model gpt-4oanthropic / claude-opus-5
measured at builder output, before any sdk normalisation
span 287 tokens (estimate, plus or minus 12%)
prompt 291 tokens (estimate, plus or minus 12.1%)
minimum 4,096 tokens for this model (inferred, unverified)
boundary declared boundary at block 0 of 2
FAIL GH102 cacheable span is below the minimum for this model, so nothing is
cached
confidence: certain
The span is 287 tokens, at most 322 allowing for estimation error. This model
requires 4,096. Nothing in this request is being cached.
The API does not report an error when a span falls below the minimum. The
request succeeds and every token is billed in full.
Fix: Add about 3,809 more tokens of stable content above the boundary, or use
a model with a lower minimum. Content only counts toward the minimum if it is
identical on every request.
Every finding says what happened, where, why it matters, and what to do. Cost appears as the unit that makes a finding legible, per request, never as a dashboard.
A request enters from one of three places: captured at the wire in a test, built by a function the check calls, or read from a saved file or log. A provider adapter parses it into one canonical form, a list of ordered blocks and parts. The core hashes that form, locates the first byte where two prefixes diverge, and measures the shared span. The classifier decides whether the span qualifies to be cached and whether it stays identical, and names the cause when it does not. The report only renders.
flowchart TB
cap[Capture at the wire]
build[Builder function]
file[Saved request or NDJSON log]
cap --> adapt
build --> adapt
file --> adapt
adapt[Provider adapter parses the body] --> ir[Canonical prefix, ordered blocks and parts]
ir --> core[Core hashing, spans, first divergence, diff]
core --> classify[Classify, does it qualify and is it stable]
classify --> report[Report renders, decides nothing]
report --> out[Finding names a cause and a fix]
table[(providers.json, facts with confidence and a date)] -. profile as a value .-> core
table -. profile as a value .-> classify
The provider facts flow in as a resolved value, never as an import. The core and
the classifier never learn a provider name, which is enforced by a test, so
adding a provider is a row in data/providers.json and a small adapter, not a
refactor. That same rule is why an unknown fact makes a check skip and say so,
rather than run against a guess.
One request, whether it qualifies to be cached at all:
| Code | Finding |
|---|---|
| GH101 | No cache boundary declared, on a provider that caches nothing without one |
| GH102 | Cacheable span below the minimum for the model |
| GH103 | More cache boundaries than the provider honours |
| GH104 | Cacheable span beyond the provider lookback window |
| GH105 | Span falling between the provider cache steps |
| GH106 | No cache routing key, where hits depend on reaching the same backend |
| GH110 | The provider data used for the run is out of date |
A set of requests, whether the cacheable prefix stays identical, which is what decides whether the cache pays across real traffic:
| Code | Finding |
|---|---|
| GH120 | The cacheable prefix is not identical between runs, classified by cause: a timestamp, a uuid, a counter, a random value, an unstable order, or genuine content |
| GH130 | A conversation stops reusing its prefix at a turn, because an earlier turn was re-rendered on replay |
groundhog explain GH120 prints the full write-up for any of them, and
docs/failure-modes.md catalogues every failure mode the
tool detects, with the cause and the fix for each.
groundhog doctor request.json --model claude-sonnet-4-5 one request, does it qualify
groundhog check requests.json --model claude-sonnet-4-5 a set, is the prefix stable
groundhog chain turns.ndjson --model claude-sonnet-4-5 a conversation, per turn reuse
groundhog audit requests.ndjson real traffic, where the loss goes
The determinism check runs the builder across a matrix: repeats catch what changes on every call, varied inputs catch per request data above the boundary, and a frozen clock, shifted timezone, and reseeded randomness catch environment dependencies that would otherwise repeat inside one process and differ everywhere else. When the prefix breaks it names the byte, shows the diff, and classifies the cause.
Raw hit rate is a misleading target: even a perfect application pays one cache write per prefix per time to live window. The audit measures how far the observed rate sits below what the traffic could reach, and splits the gap by cause.
anthropic / claude-sonnet-4-6
requests 36 over 12 minutes
observed 69.4% hit rate
achievable 97.2% given this traffic and window
recoverable 27.8%, about 0.0567 USD across this log
where it goes
22.2% prefix fragmentation (8 requests, 0.0454 USD)
2.8% time to live expiry (1 request)
2.8% routing or scope (1 request)
2.8% necessary cold writes (1 request)
It reads an NDJSON log, one record per request, each carrying a timestamp, the
model, the cacheable prefix hash, and the usage the response reported. No prompt
content is needed. Build the records in production with buildAuditRecord, which
turns a request and its response usage into one log line and keeps no content:
import { auditRecordLine } from 'groundhog';
const line = auditRecordLine({
body: requestBody,
usage: response.usage,
ts: new Date().toISOString(),
});
appendFileSync('.groundhog/requests.ndjson', line + '\n');The routing or scope bucket is the honest one: when a request should have hit by our model but the usage says it did not, the miss is outside the prefix, and the audit says so rather than reporting a rate it invented.
A prompt edit shipped on a deploy invalidates every cached prefix at once, and nothing reports it. Commit a baseline hash beside the code, and a change to the cached prefix becomes a one line diff in review.
groundhog baseline request.json --model claude-sonnet-4-5 # records the hash
git add .groundhog/baseline.json # commit it
groundhog baseline request.json --model claude-sonnet-4-5 --check # fails on driftWhen the check fails, groundhog blame walks the lockfile's git history and
names the commit that changed it:
prompt
current 9c9aeb0bbf76, 750 tokens
changed 8b10152a246c on 2026-08-01, "add source citations to the system prompt"
from c23334ef9c95
That turns "our costs jumped last month and nobody knows why" into one command.
Accept an intended change with --update.
Provider caching rules are often documented loosely and change without notice.
The conformance suite sends designed requests to a real endpoint and reads the
cache usage back, so a table value can carry the confidence observed with a
date rather than reported from a blog post.
groundhog conformance run --provider anthropic --model claude-sonnet-4-5 # dry run
GROUNDHOG_API_KEY=... groundhog conformance run --provider anthropic --model claude-sonnet-4-5 --yesIt measures the real minimum cacheable length, the cache step size, and whether
trailing whitespace, tool order, or unicode form change the cache key. Add
--ttl to also measure the time to live, which waits several real minutes. It is
the only part of the tool that touches a provider, it sends real billable
requests, and it will not run without --yes. The output is a
groundhog.providers.json fragment of what it measured, for you to review and
keep.
It will not report a pass it cannot support. Token counts are estimates with an error band. When the band crosses a threshold the answer is "no verdict", not "looks fine". A wrong pass here is worse than no tool. The band is measured, not asserted: see calibration/README.md for how.
It will not guess a provider fact. Every threshold, limit, price and field
name lives in data/providers.json with a confidence level, a source and a
verification date. When a fact is unknown the check that needs it is skipped and
named, never run against a value borrowed from a similar provider. Values marked
reported came from documentation and are not yet verified; the conformance
suite above turns them into observed.
It will not go on the hot path. There is no proxy and no gateway. The recorder patches fetch inside your test process and nowhere else.
Provider caching rules change without notice. Every run prints how old its data is, and any value can be corrected without forking:
{
"schemaVersion": 1,
"staleAfterDays": 90,
"providers": {
"anthropic": {
"models": {
"claude-opus-5": {
"minCacheableTokens": { "value": 2048, "confidence": "observed" }
}
}
}
}
}Save it as groundhog.providers.json in your project root. Layers merge, so a
correction to one number leaves everything else alone. groundhog providers show anthropic claude-opus-5 prints the merged result and where each value came
from.
groundhog doctor [file] one request, does it qualify to be cached
groundhog check [file] a set of requests, is the cacheable prefix stable
groundhog chain [file] a conversation, does each turn reuse the prefix
groundhog audit <log> real traffic, how far below the ceiling and why
groundhog baseline <file> record or check the prefix hash against a lockfile
groundhog blame show which commit changed the cached prefix
groundhog conformance run measure a provider against its live API
groundhog providers list list providers in the data table
groundhog providers show <id> [model]
groundhog explain <code> full write-up for a finding code
Exit codes: 0 no failures, 1 at least one failure, 2 usage or
configuration error, 3 a check reached no verdict and --strict was set.
npm install
npm test
npm run buildDetector cases live in test/fixtures/static, one directory per case, walked by
a single table-driven test. Adding a case means adding a directory. The cases
that carry the weight are the negative ones, which are designed to make a
detector fire wrongly, because a detector that fires on everything passes every
positive case.
Terminal output is pinned by golden files. Regenerate with
GROUNDHOG_UPDATE_GOLDEN=1 npm test and read the diff before committing it.
Output is part of the product, so a change to it should be reviewed rather than
absorbed.
MIT