Deterministic emulator timing, stage profile v2, and experimental overlap model

[qichao-arlo-wang]

Emulator-core half of #97, split out so it can land on its own. #97 keeps the
moe_timing/ Python harnesses and will be rebased onto this once it merges.

Self-contained: 13 files, no reference to moe_timing/, no dependency on the
Python layers in #97.

What this does

1. Deterministic timing

Timing measurement is only meaningful if the same artifact produces the same
cycle count twice. Two sources of non-determinism are fixed:

• lib/runtime/src/executor.rs — timers scheduled for the same instant were
  tie-broken by comparing their heap addresses (core::ptr::eq / (self as *const Self).cmp(...)). Allocation layout varies between runs, so same-instant
  events could fire in different orders. Replaced with a monotonic
  sequence_id issued by ExecutorInner, making the tie-break allocation-independent
  creation order. New test: test_same_instant_events_fire_in_schedule_order.

• lib/memory/src/chunked.rs — gather() raced all chunk reads in a
  FuturesUnordered pool. Completion order genuinely doesn't matter (each result
  carries its own dst_offset), but issue order reaches ramulator's FR-FCFS
  scheduler and changes row-hit/miss behaviour, and therefore timing. Switched to
  join_all over an ordered iterator: reads are issued in input order, then
  complete concurrently. New test: test_gather_issues_reads_in_input_order
  asserts the recorded address sequence. lib/ramulator/src/model.rs gets the
  same treatment.

run_emulator_repeat_gate() (emulator_runner.py) is the opt-in check: run the
same artifact N times, require identical sim_latency_cycles, fail otherwise.

2. Stage profile schema v1 → v2

stage_profile.rs previously recorded only instructions / seconds / HBM bytes.
Added:

• wall_cycles per opcode, and resource_proxy_cycles bucketed by opcode family
  (matrix / vector / scalar / dma / other). The buckets are disjoint —
  every opcode lands in exactly one — so a total is their plain sum.
• Cycle-accounting self-check fields: total_simulation_cycles,
  total_profiled_cycles, total_stage_wall_cycles, total_unprofiled_cycles,
  cycle_accounting_status.
• logical_bytes_* (explicitly null) split from physical_hbm_bytes_* (real
  64B HBM deltas), each with a status string, so a reader can't mistake one for
  the other.
• Classification coverage guard. Stage labels come from grepping the
  compiler's generated ASM comments — an implicit cross-repo contract. If the
  compiler's comment vocabulary drifts, opcodes silently fall into Other and
  the profile is quietly wrong. The profiler now emits
  classification.unclassified_fraction and warns past a threshold, and
  gpt_oss_moe_expert_test.py (already in CI) asserts < 0.35. Measured 12.2%,
  so there is ample margin. Drift now turns CI red instead of corrupting data.

3. Experimental prefetch/compute overlap model (off by default)

do_ops is still strictly serial. timing_overlay.rs is a post-hoc estimator
layered on top of that serial execution, gated behind
--experimental-overlap-prefetch-compute. It tracks pending prefetches with
their SRAM write ranges; each compute op first retires prefetches whose write
range overlaps its read range (a real dependency), then hides the remaining
independent prefetch time behind its own duration.

Two correctness details worth calling out:

• Time is accumulated in picoseconds and converted to cycles exactly once in
  summary(). Accumulating per-op div_ceil cycles systematically over-hides —
  two 500 ps prefetches would each round up to a full cycle. Covered by
  sub_cycle_prefetches_hide_in_picosecond_domain_not_per_op.
• M_*_WO write-out ops carry only a destination (rd + imm), no input
  operands, so they emit an empty read set. Treating their output region as a
  read would spuriously retire prefetches as false dependencies.

This changes reported cycles only — never functional execution or HBM traffic.

Known limitations (deliberate, documented in-code)

• The overlay is an estimate, not a measurement. It does not model DMA bandwidth,
  queue depth, or bank contention; and H_STORE_V / C_BREAK conservatively
  flush all pending prefetches. Real concurrent prefetch execution in the
  transactional sim is follow-up work.
• timing_access_for_opcode hand-mirrors each opcode's address computation
  (verified against the dispatch arms today). There is no shared source of truth
  between the two, and no test that they agree.
• StageProfiler::duration_to_cycles rounds per-op, so total_stage_wall_cycles
  and total_simulation_cycles agree only while every op duration is a whole
  multiple of PERIOD. That holds today (hbm2_preset, tCK = 1 ns = PERIOD);
  a different DRAM preset or core clock would break the identity.
  cycle_accounting_status reports which case you are in.
• Stage classification remains string-matching on compiler ASM comments. The
  coverage guard detects drift; it does not remove the coupling. Replacing it
  with a structured stage map emitted by the compiler needs a PLENA_Compiler PR.

Behaviour change affecting existing tests

run_emulator() now runs cargo build --release on every call instead of
only when the binary is missing. This prevents false failures where newly
generated ASM hits a stale release binary with old opcode decode logic. The
build is a fast no-op when current, but it applies to every existing caller
(aten/compare/, models/gpt_oss/, routed_moe/, …), so local test runs pick
up an extra cargo check.

Verification

• cargo build --release — clean, no warnings
• cargo test --workspace --release — 142 passed, 0 failed
• cargo fmt --all -- --check — clean
• uv run ruff format --check . / ruff check . — clean
• just test-routed-moe-expert — PASSED, 100% match rate, classification
  coverage 12.2% unclassified
• Emitted stage_profile.json verified by hand: schema_version: 2, resource
  buckets sum exactly to total_profiled_cycles (4516), cycle_accounting_status: profiled_cycles_match_total

🤖 Generated with Claude Code