Skip to content

bench: add hosted MCP workload proof - #307

Merged
lbliii merged 1 commit into
mainfrom
codex/issue-229-mcp-hosting
Jul 10, 2026
Merged

lbliii merged 1 commit into
mainfrom
codex/issue-229-mcp-hosting

Conversation

@lbliii

@lbliii lbliii commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

Add the first mergeable slice of #229: an artifact-ready, real-Milo MCP workload plus Pounce-owned transport proof.

  • extend the fixed-rate driver to rotate typed request variants with distinct bodies and headers
  • add a modern tools/call workload alternating a pure-Python CPU tool and a blocking tool
  • run the same CLI.asgi_app() under Pounce, Uvicorn, Hypercorn, or Granian through the existing Epic: Benchmark evidence program — sustained load, p99, published artifacts #228 artifact pipeline
  • require the coordinated-omission-safe --rate path instead of silently sending a different request through wrk/hey
  • prove real Pounce workers preserve MCP routing/auth header values and complete multi-event SSE framing
  • preserve existing --workload all behavior unless the fixed-rate driver is selected

Progresses #229. This deliberately does not close the epic.

Dependency and deployment blockers

The workload uses Milo's public CLI.asgi_app() from milo-cli#127. Until that draft stack merges and releases, install that branch in the benchmark environment; the benchmark app fails with an actionable message on older Milo releases.

The existing Railway pounce-railway-smoke service is healthy and remains untouched. #229's Milo MCP deployment should be a separate service rather than repurposing the production canary; provisioning that service is intentionally deferred pending explicit infrastructure/cost approval and the Milo release.

Milo #107 still owns the end-user Railway/Cloud Run/Fly recipes. Pounce #248 already supplies the free-threaded Railway $PORT/healthcheck base those recipes can copy.

Proof

  • hosted PR matrix: every required lane passed, including Linux/macOS 3.14t, coverage, process workers, framework compatibility, Railway recipe container, types/lint, and issue-closure acceptance
  • full suite: 2,425 passed, 7 skipped, 10 benchmark tests deselected
  • focused issue suite: 44 passed
  • Ruff lint and format: clean
  • silent-exception, raise-message, and import-boundary gates: clean
  • ty check src/pounce/: clean
  • Python 3.14t with PYTHON_GIL=0: the local Milo Investigate and add a measured sendfile minimum-size threshold so tiny static files skip transport-detach overhead #127 app completed the alternating workload under both Pounce and Uvicorn with no request errors
  • emitted smoke artifacts validated against benchmarks/artifact-schema.json; no local number is promoted as a public claim

Performance note

No Pounce parser, worker, scheduler, framing, or request-pipeline hot path changed. The implementation is benchmark-driver and fixture code only, so no before/after runtime benchmark is required. Short local runs validated executability and artifact shape; publishable comparison numbers still require the repeated hosted 3.14/process and 3.14t/thread matrix described by #229.

Steward notes

  • Performance evidence: alternating bodies/headers retain fixed-rate scheduling, p50/p99/p999, raw output, telemetry, and schema-compatible artifacts.
  • ASGI/protocol: real-worker tests verify header values and ordered multi-event SSE bytes without changing the HTTP implementation.
  • Security/exposure: the benchmark uses an explicit public fixture token and exact Origin allowlist; no real credential is stored.
  • Free-threading: the CPU handler is pure Python and the blocking handler models synchronous external work; the same app/request plan is used for every server comparison.
  • Public claims: README names the upstream draft dependency and forbids treating pre-release/local runs as product evidence.
  • Collateral: benchmark docs, issue-linked unit/integration proof, and changelog fragment move together; no runtime API or config collateral is needed.

@lbliii
lbliii marked this pull request as ready for review July 10, 2026 00:20
Copilot AI review requested due to automatic review settings July 10, 2026 00:20

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@lbliii
lbliii merged commit 01fa0d6 into main Jul 10, 2026
12 checks passed
@lbliii
lbliii deleted the codex/issue-229-mcp-hosting branch July 10, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants