refactor[next]: give the build caches one home too - #2792
Draft
havogt wants to merge 1 commit into
Draft
Conversation
The translation caches now share a parent, but the build caches were still
loose in the cache base, one folder per compiled program variant sitting next
to `translation_cache/` — so the base mixed a fixed directory with an unbounded
set of program folders, and neither said which kind of cache it was.
Put them under `build_cache/`, so the base holds exactly the two kinds of cache
gt4py.next keeps and each says what it is:
.gt4py_cache/
├── translation_cache/{dace,gtfn}/
└── build_cache/<program>_pyext_<fingerprint>_<version>/
No backend level here: which backend produced a build folder is not knowable
where the folder is named, since `get_cache_folder` is reached from the
build-system layer, which is generic over the source language rather than aware
of backends. The fingerprint already keeps variants apart, so the flat set is
correct — only its location changes.
`dace_determinism` scans that directory and so far learned only the folder-name
pattern from the gt4py under test. Extend that probe to the directory name as
well, rather than hardcoding a second literal: a gt4py-side move of the build
cache now surfaces as a fetch error rather than as "no programs observed".
Entries under the previous layout are not migrated, and build caches are the
large ones. They are already discarded whenever the build-cache version
changes, so the stale directories can simply be removed.
havogt
force-pushed
the
havogt/next-build-cache-dir
branch
from
August 20, 2026 11:27
52a904d to
62795cc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Top of the stack: #2769 (layout names) → #2768 (cache manager CLI) → this.
Why
#2769 gave the two translation caches one home. The build caches were left
loose in the cache base — one folder per compiled program variant, sitting
directly next to
translation_cache/:So the base mixed the two, and nothing in the listing said which entry was
which kind of cache. That is also what invites "is the DaCe build directory
inside
translation_cache?", the question that started #2769.What
The base now holds exactly the two kinds of cache gt4py.next keeps, each named
for what it is.
No
<backend>level here, deliberately. Which backend produced a buildfolder is not knowable where the folder is named:
get_cache_folderis reachedfrom
otf/compilation/and its build systems, which are generic over the sourcelanguage rather than aware of backends, so a backend level would mean either
threading a name through that layer or mapping language→backend. The fingerprint
already keeps variants apart, so the flat set is correct — only its location
changes.
dace_determinism.pyscans that directory and so far learned only thefolder-name pattern from the gt4py under test. Its probe now returns the
directory name too, rather than gaining a second hardcoded literal, so a
gt4py-side move of the build cache surfaces as a fetch error instead of as "no
programs observed". The stub interpreters in its tests answer with a different
directory name than gt4py's own, which is what proves nothing is hardcoded.
Migration
Not migrated — and these are the large caches. They are already discarded on any
change of
BUILD_CACHE_VERSION_ID(every gt4py version bump, and every commitfor an editable install), so orphaned build folders are the normal state of this
cache; the stale directories can simply be removed.
Verification
resulting tree, then read it back through
gt4py-next-cache pathandlist --by-program.dace_determinism checkagainst real caches in the new layout: it reports3 in run1, 3 in run2, i.e. it locates the program folders one level down.Before this change that scan returns 0 and raises
NoProgramsObservedError.scripts/test python: 46 passed.tests/next_tests/unit_tests/test_gt_cache_manager.pyand
otf_tests: 137 passed. Hooks clean.build-folder name but the wrong parent is still refused.