Skip to content

Add compiler-derived compute and HBM latency models - #103

Open
YuxuanHan0326 wants to merge 10 commits into
mainfrom
yx/upstream-analytical-latency-v1
Open

Add compiler-derived compute and HBM latency models#103
YuxuanHan0326 wants to merge 10 commits into
mainfrom
yx/upstream-analytical-latency-v1

Conversation

@YuxuanHan0326

Copy link
Copy Markdown
Collaborator

Summary

Adds a compiler-derived analytical latency path covering both compute execution and HBM service.

Data flow

Final compiler schedule
→ symbolic CostTrace
→ compute timing
→ physical DMA traffic
→ HBM V4
→ stage-wise roofline latency

Main changes

  • Compiler-trace compute timing providers
  • Stage, resource, opcode, and operand-variant breakdowns
  • Configured main timing and explicit experimental ideal-II1 timing
  • Generic HBM V4 physical-request model
  • Exact affine sufficient-statistics aggregation
  • Stage-wise compute/memory roofline
  • Semantic caching and compressed long-context evaluation

Validation

  • Detailed and compressed compute/DMA totals are identical
  • HBM grouped and scalar backends preserve traffic exactly
  • HBM V4 holdout median error: 3.48%
  • HBM V4 holdout P95 error: 18.10%
  • Qwen3-32B 90k×8 cold evaluation: 19.31 s
  • Long-context warm evaluation is sub-millisecond with a semantic cache hit

Dependency

Depends on PR AICrossSim/PLENA_Compiler#71

The Compiler branch commit is already available in the official repository, so this PR can be reviewed and tested before the Compiler PR is merged.

Compatibility

The existing closed-form latency backend remains available and unchanged.

Hash model, settings, compiler sources, and timing inputs before reusing a report. Keep a bounded four-entry process LRU so repeated DSE evaluations avoid rebuilding symbolic schedules while changed inputs invalidate deterministically.
Use Python 3.12 collection protocols and direct future annotations so the standalone CostEmitter branch passes its package lint checks.
Normalize public exports and Python 3.12 collection annotations so the generic memory backend is lint-clean when reviewed independently.

@GeorgeWu1204 GeorgeWu1204 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, ty

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants