Skip to content

Hybrid Router - RouterArena Submission#150

Merged
yl231 merged 1 commit into
RouteWorks:mainfrom
mikemao27:hybrid-router
Jun 20, 2026
Merged

Hybrid Router - RouterArena Submission#150
yl231 merged 1 commit into
RouteWorks:mainfrom
mikemao27:hybrid-router

Conversation

@mikemao27

Copy link
Copy Markdown
Contributor

Relevant Files

  • hybrid-router.json- regular predictions
  • hybrid-router-robustness.json - robustness predictions
  • hybrid-router.json (config) - Router configuration

Submission Steps

  1. Fork https://github.com/RouteWorks/RouterArena
  2. Copy Files:
    • router_inference/predictions/hybrid-router.json
    • router_inference/predictions/hybrid-router-robustness.json
    • router_inference/config/hybrid-router.json
  3. Open PR to RouteWorks/RouterArena
  4. Comment /evaluate

Expected Performance

  • Accuracy: ~71.35%
  • Robustness Score: ~0.9667
  • Routing Distribution: ~98.85% to qwen3-235b-a22b, ~3.4% to ministral-3b, the remainder to qwen3-30b-a3b.

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

10 similar comments
@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@mikemao27

Copy link
Copy Markdown
Contributor Author

/evaluate

@github-actions

Copy link
Copy Markdown

Router Evaluation Results

Router: hybrid-router
Dataset Split: full

RouterArena Metrics

Metric Value
RouterArena Score 0.7208
Accuracy 71.38%
Total Cost $0.321511
Avg Cost per Query $0.000038
Avg Cost per 1K Queries $0.0383
Number of Queries 8400
Abnormal Entries 0
Robustness Score 0.9667

Optimality Metrics

Metric Value
Opt.Sel (Optimal Selection) 0.8987
Opt.Cost (Cost Efficiency) 0.9419
Opt.Acc (Accuracy vs Optimal) 0.9281

Evaluation completed by RouterArena automated workflow

@mikemao27

Copy link
Copy Markdown
Contributor Author

@yl231 Thanks for your feedback on my previous router submission. I think it should be good to go this time. Would be good to note that for OpenRouter users, mistralai/ministral-3-3b-2512 now goes by mistralai/ministral-3b-2512. The model costs should still remain the same.

@yl231 yl231 merged commit 89766a0 into RouteWorks:main Jun 20, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants