Context
AISimulate PR #80 introduces Stage 1 DeepEP low-latency decode modeling. Local expert compute remains measurement-backed through MoeExpertCompute, including the SGLang DeepEPMoE.run_moe_core path. The deterministic Monte Carlo model applies only to DeepEP-LL dispatch/combine endpoint imbalance.
This non-blocking follow-up owns the operational cost of that communication model after Stage 1 lands.
Follow-up
- Benchmark uncached donor Monte Carlo queries for representative EP8/32/64/128 configurations in an optimized production build.
- Measure end-to-end
predict and recommend cost across realistic candidate counts, including cache-hit rate.
- Set an acceptable cold-query and sweep-time budget.
- If the budget is exceeded, evaluate adaptive trial counts, precomputation, or broader cache reuse while preserving deterministic results.
Acceptance criteria
- A reproducible optimized benchmark reports cold and warm timings for representative exact and donor queries.
- The measured
predict/recommend impact and agreed performance budget are documented.
- Remediation work is filed if the budget is missed.
Related: #80
Context
AISimulate PR #80 introduces Stage 1 DeepEP low-latency decode modeling. Local expert compute remains measurement-backed through
MoeExpertCompute, including the SGLangDeepEPMoE.run_moe_corepath. The deterministic Monte Carlo model applies only to DeepEP-LL dispatch/combine endpoint imbalance.This non-blocking follow-up owns the operational cost of that communication model after Stage 1 lands.
Follow-up
predictandrecommendcost across realistic candidate counts, including cache-hit rate.Acceptance criteria
predict/recommendimpact and agreed performance budget are documented.Related: #80