Skip to content

[feature] add a native Python host driver #683

Description

@dnandakumar-nv

Problem

Python applications that embed Switchyard often already own the provider client, retries, deadlines, and accounting. Today, they must drive run_stream themselves, schedule each provider call, and handle callback failures, cancellation, and stream cleanup. That repeats the same orchestration in every integration and makes lifecycle mistakes easy.

Proposed solution

Add a small Python entry point:

outcome = await switchyard.libsy.drive(algorithm, request, serve, headers=headers)

drive should run the existing Rust driver and call the host-supplied async serve(call) function for each model call. The callback completes the call with respond(...) or fail(...), and can attach the served model and provider receipt. The driver should return the normal RoutingOutcome, including an accepted response and its source metadata when routing accepts one.

The host adapter stays responsible for provider-specific work. Switchyard owns the routing loop, callback scheduling, cancellation, and native stream cleanup.

Alternatives considered

  • Keep asking every Python integration to consume run_stream directly. This is flexible, but duplicates lifecycle code and error handling.
  • Add a separate wrapper for each host integration. This would repeat the same behavior and make cleanup rules differ between integrations.
  • Use the standalone server or a Rust host. That does not fit applications that need an in-process Python integration with an existing client.

Scope notes

  • Owning surface: Python binding, with a thin PyO3 bridge to the existing Rust driver.
  • Public interface: Adds a public Python drive(...) API and response source metadata on model calls and outcomes.
  • Compatibility: Existing run_stream callers remain supported. No server API, routing configuration, or protocol type changes are required.

Additional context

Related implementation: PR #667.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions