feat[next]: cache manager CLI for the translation caches - #2768
Draft
havogt wants to merge 1 commit into
Draft
Conversation
This was referenced Aug 10, 2026
havogt
force-pushed
the
havogt/next-cache-manager
branch
from
August 10, 2026 14:05
21dee6b to
6b0aee8
Compare
havogt
force-pushed
the
havogt/next-cache-manager
branch
from
August 10, 2026 14:16
6b0aee8 to
42dc7ae
Compare
havogt
force-pushed
the
havogt/next-cache-manager
branch
3 times, most recently
from
August 10, 2026 15:23
8b28c6b to
c738cdd
Compare
havogt
force-pushed
the
havogt/next-cache-manager
branch
from
August 10, 2026 15:27
c738cdd to
17bd2a2
Compare
havogt
force-pushed
the
havogt/next-cache-manager
branch
from
August 14, 2026 13:45
17bd2a2 to
d329452
Compare
gt4py.next keeps the output of the translation step -- the optimized SDFG for DaCe, the generated sources for gtfn -- in a cache keyed by the program and the gt4py version, not by the gt4py sources. Editing a transformation or an optimization pass therefore leaves the key intact, so the next run replays the cached translation and the pass never runs, while the build step still recompiles from it and refreshes the library's mtime. Clearing only the build folder and seeing a rebuilt library looks like proof that the change took effect. It is not, and reading it that way once cost four multi-node benchmark jobs. Add `gt4py-next-cache`, mirroring `gt4py.cartesian.gt_cache_manager`: stdlib argparse, library functions plus a `__main__` block. `path` resolves the cache directories for the current environment, which is worth asking first since the default session lifetime puts them in a temporary directory that is gone when the process exits. `list` shows the entries, or with `--by-program` one row per program next to its build folders -- where a cached translation without a usable build folder is called out, since that combination is the trap. `delete` removes entries by key, program glob or all of them, after listing what matched and asking; it only prompts on a terminal and otherwise requires `--yes`, so a job file neither blocks nor proceeds unasked, and it takes the same lock as the runtime because these caches are shared between ranks. Nothing here predicts. A hit is decided by a fingerprint taken at run time over the lowered program and its arguments, which no tool outside a run can compute, and the caches record only a program name. Counts are reported as counts: no entries for a program means it will be re-translated; entries present is a reason to delete, never evidence that a run replayed. Build folders are the exception, since their name records the build-cache version, so those no run here can hit are counted as stale. Whether a build also ran to completion is not checked: it would mean reading each folder, and in a real cache all 51 build folders were complete while every stale one was stale by version. Entries are read directly rather than through `FileCache`, whose `__getitem__` deletes what it cannot unpickle -- an inspection tool must not do that -- so a corrupt or version-skewed entry degrades to `<unreadable>` instead of crashing. `[project.scripts]` gains the distribution's first console script, so the tool is reachable wherever gt4py is installed, including an icon4py environment on a cluster, which is where this failure mode bites. It is named after the subpackage because `gt4py.cartesian` has a separate cache with its own manager. The accompanying skill states the anti-pattern for agents: a fresh library mtime is evidence of recompilation, never of re-translation. It is the second skill in the repo, so AGENTS.md now records where skills live -- and that Claude Code reaches them through the tracked `.claude/skills` symlink, which fails silently if that link is ever dropped.
havogt
force-pushed
the
havogt/next-cache-manager
branch
from
August 14, 2026 13:49
d329452 to
a822bb1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2769, which names the cache layout this builds on. This PR adds a
module and touches no existing gt4py code — the new CLI, its tests, the
console-script entry, an agent skill, and one line in
AGENTS.mdrecording whereskills live (this is the repo's second one) and that Claude Code reaches them
through the tracked
.claude/skillssymlink, which fails silently if dropped.Why
gt4py.nextkeeps two persistent caches side by side under the cache base:<program>_pyext_<hash>_<version>/translation_cache/<backend>/The translation cache key covers the program and
gt4py.__version__, not thegt4py sources. Editing a transformation or an optimization pass therefore leaves
the key intact (an editable install only changes its version on a commit — the
dirty marker is a constant
.dirtysuffix — and a non-editable install neverdoes). The next run replays the cached translation and skips the pass, while the
build step still recompiles from it and refreshes the library's mtime.
That combination is easy to misread: clearing only the
*_pyext_*folder andobserving a rebuilt library looks like proof that a changed pass ran, when it
never executed. It cost four multi-node benchmark jobs and about a day of
investigation once, with two further tells misread along the way (successive SDFG
dumps differing only in
guidfields, and debug logging inside the passproducing no output).
What
gt4py-next-cache(alsopython -m gt4py.next.gt_cache_manager), mirroring theexisting
gt4py.cartesian.gt_cache_manager: stdlibargparse, library functionsplus a
__main__block, no new dependency.--backendnarrows to one cache,--cache-dirpoints at another cache base.path— the cache directories this environment resolves, honouringGT4PY_BUILD_CACHE_DIR/GT4PY_BUILD_CACHE_LIFETIME, and the build foldernaming scheme beside them. Worth asking first: the default session lifetime
puts the cache in a temporary directory that is gone when the process exits.
list— the entries: backend, key, program, size, mtime;--filter,--sort,--json. With--by-program, one row per program plus its buildfolders, which is where the two-cache trap becomes visible.
--fail-if-cachedexits non-zero if anything matched, to gate a job on anempty cache.
delete— by--key,--program <glob>or--all. Lists what matched,then asks
[y/N]; per clig.dev it only prompts when stdinis a terminal and otherwise requires
--yes, so a job file never blocks on aprompt (nor crashes on EOF the way
pip uninstalldoes).-n/--dry-runpreviews. Takes the same
locking.lockas the runtime (these caches are sharedbetween MPI ranks) and leaves build folders alone unless
--include-build-dirs.What it deliberately does not do: predict. A cache hit is decided by a
fingerprint taken at run time over the lowered program and its arguments
(
CachedStep.cache_key), which no tool outside a run can compute. The cachesrecord only a program name, so everything here is a count of what is on disk.
Read in the one direction that holds: no entries for a program means it will be
re-translated; entries present is a reason to delete, never evidence that a run
replayed. Build folders are the exception — their name records the build-cache
version, so folders no run here can hit are counted as stale. Whether a build
also finished is not checked: it would mean reading every folder, and measured
on a real cache all 51 build folders were complete while every stale one was
stale by version. That keeps this PR to a single new module.
Entries are read directly rather than through
FileCache, whose__getitem__deletes what it cannot unpickle — an inspection tool must not do that. A corrupt
or version-skewed entry degrades to
<unreadable>instead of crashing.[project.scripts]gains the distribution's first console script, so the toolis on
PATHwherever gt4py is installed — including an icon4py environment on acluster, which is where this failure mode actually bites. Named after the
subpackage rather than
gt4py-cachebecausegt4py.cartesianhas a separatecache with its own manager. Happy to add an ADR for the new packaging surface if
reviewers want one.
The agent skill (
.agents/skills/translation-cache/) fires before benchmarking acodegen change, states the anti-pattern explicitly (a fresh library mtime is not
evidence a pass ran), and gives the delete/verify recipe plus the
GT4PY_BUILD_CACHE_VERSION_IDalternative.Verification
ProgramSourcepayloads throughFileCache:path resolution under both lifetimes, empty/missing/populated/corrupt caches,
grouping and staleness, the confirmation flow (terminal yes/no, EOF, and that a
non-interactive run never prompts), delete dry-run vs
--yes, refusal to touchanything outside the cache, and that build folders survive a delete without
--include-build-dirs.tests/next_tests/unit_tests: 2211 passed.mypy src/andpre-commitclean.
version, and end to end with actual compiles on both backends.
The behavioural claims were measured, not assumed, and two of them corrected the
tool: a re-run with both caches warm rebuilds nothing (26 identical
.somtimes,184s → 26s), and editing a program's source re-translates it although its entries
still sit there under the unchanged program name.