Make Compiler scripts and tests portable across checkouts - #68
Closed
Happymic wants to merge 5 commits into
Closed
Conversation
Happymic
force-pushed
the
agent/portable-compiler-tooling
branch
from
July 25, 2026 17:34
29fdbc1 to
1a26215
Compare
Restore the TVM interpreter in the TileLang run instructions, tighten the GPT-OSS router version dispatch, and put both under CI. TileLang docstrings (13 files): - Replacing the developer-specific path with a bare `python` dropped the signal that these tests need the Python 3.11 TVM venv (see tilelang_tvm_compiler/__main__.py) and its LD_LIBRARY_PATH="" convention (see test_helper.py). Use the repo-relative `.venv-tvm/bin/python` form instead, which is portable and keeps both constraints. - Apply the same form to test_loop_slice.py, which still carried a redacted absolute PYTHONPATH, so the directory is now consistent. - Drop PYTHONPATH entirely: `-m` puts the repository root on sys.path. aten/tests/test_gpt_oss_moe_reference.py: - transformers 4.x never exposes raw router logits, so recomputing them with F.linear from the router's own weights made the router_logits assertion compare golden's matmul against an identical local matmul. Skip that assertion on the 4.x path and cross-check the dense score matrix against HF's actual output instead. - Guard the tuple type before dispatching on arity: len() on a bare Tensor return yields the token count and could match an arm. - Drop the squeeze(0) on the MLP scores; GptOssMLP.forward returns the same 2D (tokens, num_experts) matrix the router produced, so the squeeze was a no-op that would break for a single-token fixture. CI: - Neither aten/tests nor assembler/tests ran in CI, so the transformers compatibility this test provides was unverified while `pip install transformers` stays unpinned. Run the two suites this PR touches. The golden_b tests are excluded because they import plena_quant from the sibling PLENA_Tools checkout, absent in this workflow. - Register the `slow` marker that -m 'not slow' relies on. assembler/parser.py: - Add help text for asm_file and report a missing file through ArgumentParser.error instead of an unhandled FileNotFoundError. tilelang_tvm_compiler/scripts/run_flash_attention_midir.py: - Drop PYTHONPATH=compiler, which assumed a superproject layout. Verified with pytest against transformers 5.14.1 (three-value router) and 4.57.1 (two-value router), in a checkout with the sibling repository removed: 13 passed, 2 deselected on both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Instruction.__init__ accepted an rflag argument but never assigned it, storing self.rmask = rstride instead, while __repr__ reads self.rflag. Every repr(Instruction) therefore raised AttributeError, so the `python assembler/parser.py <file>` entry point crashed on its first printed instruction. Pre-existing on main and out of the scope of this branch, but it blocks the entry point this branch exists to make runnable, and `--help` alone does not reach the failing path. Kept as a separate commit so it can be dropped independently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit wires this file into the unit-tests job, so the stage-attribution guard now flags its exemption entry as stale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
qichao-arlo-wang
force-pushed
the
agent/portable-compiler-tooling
branch
from
August 3, 2026 16:24
482de00 to
f8f96da
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
asm_filecommand-line argument.PYTHONPATHexamples in 12 TileLang test docstrings with repository-relative module invocations.Why
The checked-in entry point and test examples referenced individual developers' home directories. They could not be reused from another checkout. The GPT-OSS test also depended on one specific Transformers API shape even though both shapes are still encountered in supported development environments.
Scope
The parser change affects only its direct
__main__entry point; library callers ofparse_asm_fileare unchanged. The TileLang changes are documentation-only. The GPT-OSS change is confined to the Hugging Face cross-check in one test.Validation
PYTHONPATH=. python -m pytest -q aten/tests -m 'not slow': 60 passed, 1 deselected.PYTHONPATH=. python -m pytest -q assembler/tests: 4 passed.PYTHONPATH=. python -m pytest -q aten/tests/test_gpt_oss_moe_reference.py: 11 passed with Transformers 4.57.6.python assembler/parser.py --help: command-line interface renders correctly.git diff --check: passed.The TileLang suite was not executed locally because this checkout's Python environment does not include TVM. The only changed lines in those files are the
Run:docstrings; no TileLang test logic changed.