Skip to content

feat[next]: cache manager CLI for the translation caches - #2768

Draft
havogt wants to merge 1 commit into
havogt/next-cache-dir-renamefrom
havogt/next-cache-manager
Draft

feat[next]: cache manager CLI for the translation caches#2768
havogt wants to merge 1 commit into
havogt/next-cache-dir-renamefrom
havogt/next-cache-manager

Conversation

@havogt

@havogt havogt commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Stacked on #2769, which names the cache layout this builds on. This PR adds a
module and touches no existing gt4py code
— the new CLI, its tests, the
console-script entry, an agent skill, and one line in AGENTS.md recording where
skills live (this is the repo's second one) and that Claude Code reaches them
through the tracked .claude/skills symlink, which fails silently if dropped.

Why

gt4py.next keeps two persistent caches side by side under the cache base:

Cache Where Holds
Build cache <program>_pyext_<hash>_<version>/ build artifacts, the compiled library
Translation cache translation_cache/<backend>/ the already translated program: optimized SDFG / generated sources

The translation cache key covers the program and gt4py.__version__, not the
gt4py sources. Editing a transformation or an optimization pass therefore leaves
the key intact (an editable install only changes its version on a commit — the
dirty marker is a constant .dirty suffix — and a non-editable install never
does). The next run replays the cached translation and skips the pass, while the
build step still recompiles from it and refreshes the library's mtime.

That combination is easy to misread: clearing only the *_pyext_* folder and
observing a rebuilt library looks like proof that a changed pass ran, when it
never executed. It cost four multi-node benchmark jobs and about a day of
investigation once, with two further tells misread along the way (successive SDFG
dumps differing only in guid fields, and debug logging inside the pass
producing no output).

What

gt4py-next-cache (also python -m gt4py.next.gt_cache_manager), mirroring the
existing gt4py.cartesian.gt_cache_manager: stdlib argparse, library functions
plus a __main__ block, no new dependency. --backend narrows to one cache,
--cache-dir points at another cache base.

  • path — the cache directories this environment resolves, honouring
    GT4PY_BUILD_CACHE_DIR / GT4PY_BUILD_CACHE_LIFETIME, and the build folder
    naming scheme beside them. Worth asking first: the default session lifetime
    puts the cache in a temporary directory that is gone when the process exits.
  • list — the entries: backend, key, program, size, mtime; --filter,
    --sort, --json. With --by-program, one row per program plus its build
    folders, which is where the two-cache trap becomes visible.
    --fail-if-cached exits non-zero if anything matched, to gate a job on an
    empty cache.
  • delete — by --key, --program <glob> or --all. Lists what matched,
    then asks [y/N]; per clig.dev it only prompts when stdin
    is a terminal and otherwise requires --yes, so a job file never blocks on a
    prompt (nor crashes on EOF the way pip uninstall does). -n/--dry-run
    previews. Takes the same locking.lock as the runtime (these caches are shared
    between MPI ranks) and leaves build folders alone unless --include-build-dirs.
$ gt4py-next-cache list --by-program
PROGRAM            ENTRIES         BUILDS
lap_program        3 dace, 2 gtfn  0 (+15 stale)
laplap_program     9 dace, 6 gtfn  0 (+15 stale)

WARNING: 2 program(s) have a cached translation but no usable build folder. Their
next run rebuilds the library while the cached translation is replayed, so a fresh
library there would NOT mean a changed pass ran.

What it deliberately does not do: predict. A cache hit is decided by a
fingerprint taken at run time over the lowered program and its arguments
(CachedStep.cache_key), which no tool outside a run can compute. The caches
record only a program name, so everything here is a count of what is on disk.
Read in the one direction that holds: no entries for a program means it will be
re-translated; entries present is a reason to delete, never evidence that a run
replayed. Build folders are the exception — their name records the build-cache
version, so folders no run here can hit are counted as stale. Whether a build
also finished is not checked: it would mean reading every folder, and measured
on a real cache all 51 build folders were complete while every stale one was
stale by version. That keeps this PR to a single new module.

Entries are read directly rather than through FileCache, whose __getitem__
deletes what it cannot unpickle — an inspection tool must not do that. A corrupt
or version-skewed entry degrades to <unreadable> instead of crashing.

[project.scripts] gains the distribution's first console script, so the tool
is on PATH wherever gt4py is installed — including an icon4py environment on a
cluster, which is where this failure mode actually bites. Named after the
subpackage rather than gt4py-cache because gt4py.cartesian has a separate
cache with its own manager. Happy to add an ADR for the new packaging surface if
reviewers want one.

The agent skill (.agents/skills/translation-cache/) fires before benchmarking a
codegen change, states the anti-pattern explicitly (a fresh library mtime is not
evidence a pass ran), and gives the delete/verify recipe plus the
GT4PY_BUILD_CACHE_VERSION_ID alternative.

Verification

  • 43 unit tests building real ProgramSource payloads through FileCache:
    path resolution under both lifetimes, empty/missing/populated/corrupt caches,
    grouping and staleness, the confirmation flow (terminal yes/no, EOF, and that a
    non-interactive run never prompts), delete dry-run vs --yes, refusal to touch
    anything outside the cache, and that build folders survive a delete without
    --include-build-dirs.
  • Full tests/next_tests/unit_tests: 2211 passed. mypy src/ and pre-commit
    clean.
  • Exercised against a real 224-entry icon4py cache written by a different gt4py
    version, and end to end with actual compiles on both backends.

The behavioural claims were measured, not assumed, and two of them corrected the
tool: a re-run with both caches warm rebuilds nothing (26 identical .so mtimes,
184s → 26s), and editing a program's source re-translates it although its entries
still sit there under the unchanged program name.

@havogt havogt changed the title havogt/next cache manager feat[next]: cache manager CLI for the translation caches Aug 10, 2026
@havogt
havogt force-pushed the havogt/next-cache-manager branch from 21dee6b to 6b0aee8 Compare August 10, 2026 14:05
@havogt
havogt changed the base branch from main to havogt/next-cache-dir-rename August 10, 2026 14:05
@havogt
havogt force-pushed the havogt/next-cache-manager branch from 6b0aee8 to 42dc7ae Compare August 10, 2026 14:16
@havogt
havogt force-pushed the havogt/next-cache-manager branch 3 times, most recently from 8b28c6b to c738cdd Compare August 10, 2026 15:23
@havogt
havogt force-pushed the havogt/next-cache-manager branch from c738cdd to 17bd2a2 Compare August 10, 2026 15:27
@havogt
havogt force-pushed the havogt/next-cache-manager branch from 17bd2a2 to d329452 Compare August 14, 2026 13:45
gt4py.next keeps the output of the translation step -- the optimized SDFG for
DaCe, the generated sources for gtfn -- in a cache keyed by the program and the
gt4py version, not by the gt4py sources. Editing a transformation or an
optimization pass therefore leaves the key intact, so the next run replays the
cached translation and the pass never runs, while the build step still
recompiles from it and refreshes the library's mtime. Clearing only the build
folder and seeing a rebuilt library looks like proof that the change took
effect. It is not, and reading it that way once cost four multi-node benchmark
jobs.

Add `gt4py-next-cache`, mirroring `gt4py.cartesian.gt_cache_manager`: stdlib
argparse, library functions plus a `__main__` block. `path` resolves the cache
directories for the current environment, which is worth asking first since the
default session lifetime puts them in a temporary directory that is gone when
the process exits. `list` shows the entries, or with `--by-program` one row per
program next to its build folders -- where a cached translation without a usable
build folder is called out, since that combination is the trap. `delete` removes
entries by key, program glob or all of them, after listing what matched and
asking; it only prompts on a terminal and otherwise requires `--yes`, so a job
file neither blocks nor proceeds unasked, and it takes the same lock as the
runtime because these caches are shared between ranks.

Nothing here predicts. A hit is decided by a fingerprint taken at run time over
the lowered program and its arguments, which no tool outside a run can compute,
and the caches record only a program name. Counts are reported as counts: no
entries for a program means it will be re-translated; entries present is a
reason to delete, never evidence that a run replayed. Build folders are the
exception, since their name records the build-cache version, so those no run
here can hit are counted as stale. Whether a build also ran to completion is not
checked: it would mean reading each folder, and in a real cache all 51 build
folders were complete while every stale one was stale by version.

Entries are read directly rather than through `FileCache`, whose `__getitem__`
deletes what it cannot unpickle -- an inspection tool must not do that -- so a
corrupt or version-skewed entry degrades to `<unreadable>` instead of crashing.

`[project.scripts]` gains the distribution's first console script, so the tool is
reachable wherever gt4py is installed, including an icon4py environment on a
cluster, which is where this failure mode bites. It is named after the subpackage
because `gt4py.cartesian` has a separate cache with its own manager.

The accompanying skill states the anti-pattern for agents: a fresh library mtime
is evidence of recompilation, never of re-translation. It is the second skill in
the repo, so AGENTS.md now records where skills live -- and that Claude Code
reaches them through the tracked `.claude/skills` symlink, which fails silently
if that link is ever dropped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant