Problem
Python applications that embed Switchyard often already own the provider client, retries, deadlines, and accounting. Today, they must drive run_stream themselves, schedule each provider call, and handle callback failures, cancellation, and stream cleanup. That repeats the same orchestration in every integration and makes lifecycle mistakes easy.
Proposed solution
Add a small Python entry point:
outcome = await switchyard.libsy.drive(algorithm, request, serve, headers=headers)
drive should run the existing Rust driver and call the host-supplied async serve(call) function for each model call. The callback completes the call with respond(...) or fail(...), and can attach the served model and provider receipt. The driver should return the normal RoutingOutcome, including an accepted response and its source metadata when routing accepts one.
The host adapter stays responsible for provider-specific work. Switchyard owns the routing loop, callback scheduling, cancellation, and native stream cleanup.
Alternatives considered
- Keep asking every Python integration to consume
run_stream directly. This is flexible, but duplicates lifecycle code and error handling.
- Add a separate wrapper for each host integration. This would repeat the same behavior and make cleanup rules differ between integrations.
- Use the standalone server or a Rust host. That does not fit applications that need an in-process Python integration with an existing client.
Scope notes
- Owning surface: Python binding, with a thin PyO3 bridge to the existing Rust driver.
- Public interface: Adds a public Python
drive(...) API and response source metadata on model calls and outcomes.
- Compatibility: Existing
run_stream callers remain supported. No server API, routing configuration, or protocol type changes are required.
Additional context
Related implementation: PR #667.
Problem
Python applications that embed Switchyard often already own the provider client, retries, deadlines, and accounting. Today, they must drive
run_streamthemselves, schedule each provider call, and handle callback failures, cancellation, and stream cleanup. That repeats the same orchestration in every integration and makes lifecycle mistakes easy.Proposed solution
Add a small Python entry point:
driveshould run the existing Rust driver and call the host-supplied asyncserve(call)function for each model call. The callback completes the call withrespond(...)orfail(...), and can attach the served model and provider receipt. The driver should return the normalRoutingOutcome, including an accepted response and its source metadata when routing accepts one.The host adapter stays responsible for provider-specific work. Switchyard owns the routing loop, callback scheduling, cancellation, and native stream cleanup.
Alternatives considered
run_streamdirectly. This is flexible, but duplicates lifecycle code and error handling.Scope notes
drive(...)API and response source metadata on model calls and outcomes.run_streamcallers remain supported. No server API, routing configuration, or protocol type changes are required.Additional context
Related implementation: PR #667.