Putting a dollar cap on a single agent run

[domondi1]

If you run Agent Framework agents for real users, one loop or a burst of tool calls can make a lot more model calls than you planned. As far as I can tell there's no per-run dollar limit built in yet, so here's one way to add it without changing the agent.

Point the Chat Completions client at a small local gateway and pass the run's id and budget as headers. We've been doing this with Inferrail:

from agent_framework import Agent
from agent_framework.openai import OpenAIChatCompletionClient

def agent_for_run(run_id: str, budget_usd: str) -> Agent:
    client = OpenAIChatCompletionClient(
        model="gpt-4o-mini", base_url="http://127.0.0.1:8000/v1", api_key="unused",
        default_headers={"X-Inferrail-Attribute-Work-Id": run_id,
                         "X-Inferrail-Budget-Usd": budget_usd})
    return Agent(client=client, instructions="...", tools=[...],
                 default_options={"max_tokens": 200})

Every call in that run reserves its estimated cost against the run's budget before it goes out, so parallel tool calls can't all spend the same remaining money. When a call doesn't fit, it fails with a 402 before it reaches the provider (you'll see a ChatClientException), and you can end the run there.

A few caveats: it's Chat Completions only for now, the reservation is an estimate so set max_tokens, and only calls through the gateway count. Tested with agent-framework-core 1.19.0 and agent-framework-openai 1.14.4.

Setup and the full snippet: recipe. I maintain Inferrail (open source, Apache-2.0).

[domondi1]

For .NET Agent Framework users: the same budget works by giving the agent an OpenAIClient whose HTTP transport adds the run's headers (tested with Microsoft.Agents.AI.OpenAI 1.23.0 on .NET 10). Run the gateway as a separate process (pip install inferrail, then inferrail serve --quickstart --app-mode, which listens on http://127.0.0.1:8000):

var http = new HttpClient(new RunBudgetHandler { InnerHandler = new HttpClientHandler() });
var openai = new OpenAIClient(new ApiKeyCredential("unused"), new OpenAIClientOptions
{
    Endpoint = new Uri("http://127.0.0.1:8000/v1"),
    Transport = new HttpClientPipelineTransport(http),
});
AIAgent agent = openai.GetChatClient("gpt-4o-mini").AsIChatClient()
    .AsAIAgent(instructions: "...", tools: tools);

RunBudgetHandler is a small DelegatingHandler that reads the current run's id and budget from an AsyncLocal and sets X-Inferrail-Attribute-Work-Id and X-Inferrail-Budget-Usd. Every agent.RunAsync inside using (RunBudget.Begin("run-42", "0.50")) { ... } draws from one budget, including parallel runs. A refused call surfaces as ClientResultException with Status == 402, and inferrail work run-42 shows the run's cost.

The full handler and scope code is in dotnet/extensions#7802.