Skip to content
swarinaPublic

About

Local-first prompt cache checker. Finds out why your prompt cache is not paying.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Groundhog

Find out why your prompt cache is not paying.

ci license: MIT node: 20+

Every major provider discounts a repeated prompt prefix heavily. The discount applies only when the prefix is identical to the last one, and when it is not, nothing happens: no error, no warning, a normal response, full price. Groundhog finds the reasons that does not happen, names each one, and says what to do about it.

It runs locally. Nothing leaves your machine, there is no account, and the check path makes no network calls.

Status: early. Anthropic and OpenAI are covered, plus a generic OpenAI compatible endpoint. Most provider facts, including the model minimums, are drawn from documentation and marked unverified until a conformance run measures them against the live API. Every number the tool prints carries its confidence, so you always know what is measured and what is a guess.

Contents

Install

Requires Node 20 or newer.

Not yet published to npm, so run it straight from this repository. It builds on install, so no separate build step is needed:

npx github:swarina/groundhog doctor request.json --model claude-sonnet-4-5

Or from a clone, which is also the way to use it as a library:

git clone https://github.com/swarina/groundhog
cd groundhog
npm install
node dist/cli/index.js doctor request.json --model claude-sonnet-4-5

The examples below write groundhog for the command. Read that as npx github:swarina/groundhog, or link the clone with npm link to get a real groundhog on your path.

Five minutes to an answer

Groundhog looks at the request your application actually sends, which is not always the request your code appears to build. An SDK normalises content shapes, fills in defaults and serialises before anything reaches the provider, and the provider hashes what arrived. So the accurate place to measure is the wire.

Record what your existing tests already send:

// install first: npm install github:swarina/groundhog
import { capture } from 'groundhog/capture';

const recorder = capture();

test('support agent prompt is cacheable', async () => {
  await runMyAgent({ query: 'where is my order' });

  const [report] = recorder.inspect({ model: 'claude-sonnet-4-5' });
  expect(report.ok).toBe(true);
});

Nothing is sent. The recorder answers with a minimal valid response so the code under test carries on, which means no keys, no cost and no network. If it captures nothing it raises an error rather than reporting a pass, because a check that inspected no requests has not verified anything.

If you already have a request body saved, skip the recorder:

groundhog doctor request.json --model gpt-4o

What it tells you

anthropic / claude-opus-5
measured at   builder output, before any sdk normalisation
span          287 tokens (estimate, plus or minus 12%)
prompt        291 tokens (estimate, plus or minus 12.1%)
minimum       4,096 tokens for this model (inferred, unverified)
boundary      declared boundary at block 0 of 2

FAIL  GH102  cacheable span is below the minimum for this model, so nothing is
             cached
      confidence: certain

  The span is 287 tokens, at most 322 allowing for estimation error. This model
  requires 4,096. Nothing in this request is being cached.

  The API does not report an error when a span falls below the minimum. The
  request succeeds and every token is billed in full.

  Fix: Add about 3,809 more tokens of stable content above the boundary, or use
  a model with a lower minimum. Content only counts toward the minimum if it is
  identical on every request.

Every finding says what happened, where, why it matters, and what to do. Cost appears as the unit that makes a finding legible, per request, never as a dashboard.

How it works

A request enters from one of three places: captured at the wire in a test, built by a function the check calls, or read from a saved file or log. A provider adapter parses it into one canonical form, a list of ordered blocks and parts. The core hashes that form, locates the first byte where two prefixes diverge, and measures the shared span. The classifier decides whether the span qualifies to be cached and whether it stays identical, and names the cause when it does not. The report only renders.

flowchart TB
  cap[Capture at the wire]
  build[Builder function]
  file[Saved request or NDJSON log]

  cap --> adapt
  build --> adapt
  file --> adapt

  adapt[Provider adapter parses the body] --> ir[Canonical prefix, ordered blocks and parts]
  ir --> core[Core hashing, spans, first divergence, diff]
  core --> classify[Classify, does it qualify and is it stable]
  classify --> report[Report renders, decides nothing]
  report --> out[Finding names a cause and a fix]

  table[(providers.json, facts with confidence and a date)] -. profile as a value .-> core
  table -. profile as a value .-> classify
Loading

The provider facts flow in as a resolved value, never as an import. The core and the classifier never learn a provider name, which is enforced by a test, so adding a provider is a row in data/providers.json and a small adapter, not a refactor. That same rule is why an unknown fact makes a check skip and say so, rather than run against a guess.

What it checks today

One request, whether it qualifies to be cached at all:

Code Finding
GH101 No cache boundary declared, on a provider that caches nothing without one
GH102 Cacheable span below the minimum for the model
GH103 More cache boundaries than the provider honours
GH104 Cacheable span beyond the provider lookback window
GH105 Span falling between the provider cache steps
GH106 No cache routing key, where hits depend on reaching the same backend
GH110 The provider data used for the run is out of date

A set of requests, whether the cacheable prefix stays identical, which is what decides whether the cache pays across real traffic:

Code Finding
GH120 The cacheable prefix is not identical between runs, classified by cause: a timestamp, a uuid, a counter, a random value, an unstable order, or genuine content
GH130 A conversation stops reusing its prefix at a turn, because an earlier turn was re-rendered on replay

groundhog explain GH120 prints the full write-up for any of them, and docs/failure-modes.md catalogues every failure mode the tool detects, with the cause and the fix for each.

The core checks

groundhog doctor request.json --model claude-sonnet-4-5   one request, does it qualify
groundhog check requests.json --model claude-sonnet-4-5   a set, is the prefix stable
groundhog chain turns.ndjson  --model claude-sonnet-4-5   a conversation, per turn reuse
groundhog audit requests.ndjson                           real traffic, where the loss goes

The determinism check runs the builder across a matrix: repeats catch what changes on every call, varied inputs catch per request data above the boundary, and a frozen clock, shifted timezone, and reseeded randomness catch environment dependencies that would otherwise repeat inside one process and differ everywhere else. When the prefix breaks it names the byte, shows the diff, and classifies the cause.

The audit

Raw hit rate is a misleading target: even a perfect application pays one cache write per prefix per time to live window. The audit measures how far the observed rate sits below what the traffic could reach, and splits the gap by cause.

anthropic / claude-sonnet-4-6
requests      36 over 12 minutes
observed      69.4% hit rate
achievable    97.2% given this traffic and window
recoverable   27.8%, about 0.0567 USD across this log

where it goes
   22.2%  prefix fragmentation (8 requests, 0.0454 USD)
    2.8%  time to live expiry (1 request)
    2.8%  routing or scope (1 request)
    2.8%  necessary cold writes (1 request)

It reads an NDJSON log, one record per request, each carrying a timestamp, the model, the cacheable prefix hash, and the usage the response reported. No prompt content is needed. Build the records in production with buildAuditRecord, which turns a request and its response usage into one log line and keeps no content:

import { auditRecordLine } from 'groundhog';

const line = auditRecordLine({
  body: requestBody,
  usage: response.usage,
  ts: new Date().toISOString(),
});
appendFileSync('.groundhog/requests.ndjson', line + '\n');

The routing or scope bucket is the honest one: when a request should have hit by our model but the usage says it did not, the miss is outside the prefix, and the audit says so rather than reporting a rate it invented.

Catching the deploy that cold starts the cache

A prompt edit shipped on a deploy invalidates every cached prefix at once, and nothing reports it. Commit a baseline hash beside the code, and a change to the cached prefix becomes a one line diff in review.

groundhog baseline request.json --model claude-sonnet-4-5   # records the hash
git add .groundhog/baseline.json                            # commit it

groundhog baseline request.json --model claude-sonnet-4-5 --check   # fails on drift

When the check fails, groundhog blame walks the lockfile's git history and names the commit that changed it:

prompt
  current  9c9aeb0bbf76, 750 tokens
  changed  8b10152a246c on 2026-08-01, "add source citations to the system prompt"
  from     c23334ef9c95

That turns "our costs jumped last month and nobody knows why" into one command. Accept an intended change with --update.

Measuring a provider instead of trusting its docs

Provider caching rules are often documented loosely and change without notice. The conformance suite sends designed requests to a real endpoint and reads the cache usage back, so a table value can carry the confidence observed with a date rather than reported from a blog post.

groundhog conformance run --provider anthropic --model claude-sonnet-4-5   # dry run
GROUNDHOG_API_KEY=... groundhog conformance run --provider anthropic --model claude-sonnet-4-5 --yes

It measures the real minimum cacheable length, the cache step size, and whether trailing whitespace, tool order, or unicode form change the cache key. Add --ttl to also measure the time to live, which waits several real minutes. It is the only part of the tool that touches a provider, it sends real billable requests, and it will not run without --yes. The output is a groundhog.providers.json fragment of what it measured, for you to review and keep.

Three things it will not do

It will not report a pass it cannot support. Token counts are estimates with an error band. When the band crosses a threshold the answer is "no verdict", not "looks fine". A wrong pass here is worse than no tool. The band is measured, not asserted: see calibration/README.md for how.

It will not guess a provider fact. Every threshold, limit, price and field name lives in data/providers.json with a confidence level, a source and a verification date. When a fact is unknown the check that needs it is skipped and named, never run against a value borrowed from a similar provider. Values marked reported came from documentation and are not yet verified; the conformance suite above turns them into observed.

It will not go on the hot path. There is no proxy and no gateway. The recorder patches fetch inside your test process and nowhere else.

Correcting the provider data

Provider caching rules change without notice. Every run prints how old its data is, and any value can be corrected without forking:

{
  "schemaVersion": 1,
  "staleAfterDays": 90,
  "providers": {
    "anthropic": {
      "models": {
        "claude-opus-5": {
          "minCacheableTokens": { "value": 2048, "confidence": "observed" }
        }
      }
    }
  }
}

Save it as groundhog.providers.json in your project root. Layers merge, so a correction to one number leaves everything else alone. groundhog providers show anthropic claude-opus-5 prints the merged result and where each value came from.

Command reference

groundhog doctor [file]        one request, does it qualify to be cached
groundhog check [file]         a set of requests, is the cacheable prefix stable
groundhog chain [file]         a conversation, does each turn reuse the prefix
groundhog audit <log>          real traffic, how far below the ceiling and why
groundhog baseline <file>      record or check the prefix hash against a lockfile
groundhog blame                show which commit changed the cached prefix
groundhog conformance run      measure a provider against its live API
groundhog providers list       list providers in the data table
groundhog providers show <id> [model]
groundhog explain <code>       full write-up for a finding code

Exit codes: 0 no failures, 1 at least one failure, 2 usage or configuration error, 3 a check reached no verdict and --strict was set.

Development

npm install
npm test
npm run build

Detector cases live in test/fixtures/static, one directory per case, walked by a single table-driven test. Adding a case means adding a directory. The cases that carry the weight are the negative ones, which are designed to make a detector fire wrongly, because a detector that fires on everything passes every positive case.

Terminal output is pinned by golden files. Regenerate with GROUNDHOG_UPDATE_GOLDEN=1 npm test and read the diff before committing it. Output is part of the product, so a change to it should be reviewed rather than absorbed.

Licence

MIT

About

Local-first prompt cache checker. Finds out why your prompt cache is not paying.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages