[Bench] Add external baselines for common ops

[superAngGao]

Summary

Draft PR for #1806.

• Add external baseline tags for common-op benchmarks: torch_inductor, torch_cudnn, torch_cufft, torch_native, and flashinfer.
• Add a shared torch_inductor_baseline() helper for benchmark-local PyTorch baselines compiled with torch.compile(..., fullgraph=True).
• Add FlashInfer baselines for RMSNorm, fused add RMSNorm, and semantic-equivalent RoPE variants.
• Keep optional third-party baselines local to benchmark files and preserve existing roofline reporting.
• Fix nightly report coloring to use the strongest competitive baseline when multiple baselines are present.

Validation

Official runner image: ghcr.io/tile-ai/tileops-runner:65dbc98-torch2.10.

• compileall and ruff check on changed files.
• pytest -q tests/test_nightly_report.py benchmarks/tests/test_benchmark_base.py --tb=short -> 19 passed.
• RoPE/Pool benchmark collect-only -> 52 cases collected.
• Targeted manifest checks for RoPE, RMSNorm, Conv/Pool, and FFT passed with advisory warnings only.
• TorchInductor and FlashInfer smoke checks passed.

Closes #1806.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[superAngGao]

Thanks for the careful review. I pushed a follow-up commit (1a28c9d) addressing all three points.

• Fixed RoPE FlashInfer semantics: cached FlashInfer baselines now receive variant-specific cos/sin caches. YaRN uses the benchmark-local YaRN interpolation formula, and the benchmark also checks FlashInfer output against the benchmark-local semantic reference once before timing. I also tightened the torch reference path so Neox, Non-Neox, Llama 3.1, YaRN, and LongRoPE all use semantic-equivalent references instead of geometry-only rotation.
• Renamed pooling baselines from torch_cudnn to torch_native, matching the kernels observed in the official runner. Convolution and BatchNorm remain tagged torch_cudnn.
• Fixed nightly report coloring for multi-baseline configs. Since ratio = baseline_latency / tileops_latency, the strongest competitor is the smallest ratio, not the largest. Added a small regression test for the FlashInfer=0.5 / Inductor=2.0 case.

Validation in ghcr.io/tile-ai/tileops-runner:65dbc98-torch2.10:

• compileall for the changed files
• pytest -q tests/test_nightly_report.py benchmarks/tests/test_benchmark_base.py --tb=short -> 19 passed
• pytest --collect-only -q benchmarks/ops/bench_rope.py benchmarks/ops/bench_pool.py --tb=short -> 52 benchmark cases collected
• ruff check on the changed files
• validate_manifest.py --check-op RopeYarnOp
• validate_manifest.py --check-op AvgPool2dFwdOp
• FlashInfer RoPE semantic smoke: Neox, Non-Neox, Llama 3.1, YaRN, and LongRoPE all matched the benchmark-local reference within fp16 tolerance; YaRN/Llama/LongRoPE local tables also matched the production formula helpers.

[lcy-seso]

I rebased this branch onto main (fb95f0a) and force-pushed it, so the old head 1a28c9d is gone. The branch was 56 commits behind and the benchmark API changed underneath it, so this was more than a mechanical conflict resolution — please review what landed.

What changed and why

The branch was written against bm.profile(...) + BenchmarkReport.record(..., tag=...). main has since replaced that with bm.compare({tag: functor, ...}, *inputs, record_as=, params=), which times every functor in interleaved passes and records each one under its tag. I re-expressed the two commits on the new API rather than restoring the old calls:

• torch_inductor_baseline() is kept in benchmark_base.py (plus a Callable import), and its unit test is kept.
• Every site that wrapped a baseline in TorchInductor now passes it as a compare entry: {"tileops": op, "torch-inductor": torch_inductor_baseline(fn)}. I used a hyphen to match the existing torch-ref / torch-cudnn keys on main; the nightly report keys off the tag string, so the name is free — say the word if you want torch_inductor.
• flashinfer baselines (rmsnorm, fused_add_rmsnorm, rope) are kept, recorded via bm.profile + BenchmarkReport.record(tag="flashinfer") alongside the compare call.
• bench_mean_pooling keeps your condition: the offsets path is not fullgraph-compilable, so it stays on the eager baseline.
• bench_rope keeps the variant-correct baselines and the flashinfer correctness assertions from the second commit; the torch baseline goes through compare as _rope_reference(..., variant=, interleave=).
• scripts/nightly_report.py and tests/test_nightly_report.py applied cleanly.

What I dropped

• The torch_cudnn → torch_native tag rename in bench_pool: main already names that baseline torch-ref, so there was nothing left to rename.
• The AvgPool2dBenchCase / MaxPool*BenchCase classes and the if __name__ == "__main__" blocks the diff re-added — main deleted both.
• The six BenchmarkReport config tests in benchmarks/tests/test_benchmark_base.py — main removed them when reporting moved to benchmarks/report.py. Only test_torch_inductor_baseline_uses_fullgraph_compile was carried over.

Verification

ruff clean. tests/test_nightly_report.py (1 passed), benchmarks/tests/test_benchmark_base.py (17 passed), scripts/validate_manifest.py all pass. I did not run the GPU benchmarks — CI is the real check.

One thing worth a look

flashinfer.fused_add_rmsnorm mutates x and residual in place. That was already true before, but compare runs the functors over several passes, so the tileops and torch entries will see mutated inputs. Timing-wise it may be fine; flagging it since the execution shape changed.