Skip to content

Benchmark DeepEP-LL donor Monte Carlo cost #104

Description

@jasonqinzhou

Context

AISimulate PR #80 introduces Stage 1 DeepEP low-latency decode modeling. Local expert compute remains measurement-backed through MoeExpertCompute, including the SGLang DeepEPMoE.run_moe_core path. The deterministic Monte Carlo model applies only to DeepEP-LL dispatch/combine endpoint imbalance.

This non-blocking follow-up owns the operational cost of that communication model after Stage 1 lands.

Follow-up

  • Benchmark uncached donor Monte Carlo queries for representative EP8/32/64/128 configurations in an optimized production build.
  • Measure end-to-end predict and recommend cost across realistic candidate counts, including cache-hit rate.
  • Set an acceptable cold-query and sweep-time budget.
  • If the budget is exceeded, evaluate adaptive trial counts, precomputation, or broader cache reuse while preserving deterministic results.

Acceptance criteria

  • A reproducible optimized benchmark reports cold and warm timings for representative exact and donor queries.
  • The measured predict/recommend impact and agreed performance budget are documented.
  • Remediation work is filed if the budget is missed.

Related: #80

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions