Putting a dollar cap on a single agent run

[domondi1]

If you run Agent Framework agents for real users, one loop or a burst of tool calls can make a lot more model calls than you planned. As far as I can tell there's no per-run dollar limit built in yet, so here's one way to add it without changing the agent.

Point the Chat Completions client at a small local gateway and pass the run's id and budget as headers. We've been doing this with Inferrail:

from agent_framework import Agent
from agent_framework.openai import OpenAIChatCompletionClient

def agent_for_run(run_id: str, budget_usd: str) -> Agent:
    client = OpenAIChatCompletionClient(
        model="gpt-4o-mini", base_url="http://127.0.0.1:8000/v1", api_key="unused",
        default_headers={"X-Inferrail-Attribute-Work-Id": run_id,
                         "X-Inferrail-Budget-Usd": budget_usd})
    return Agent(client=client, instructions="...", tools=[...],
                 default_options={"max_tokens": 200})

Every call in that run reserves its estimated cost against the run's budget before it goes out, so parallel tool calls can't all spend the same remaining money. When a call doesn't fit, it fails with a 402 before it reaches the provider (you'll see a ChatClientException), and you can end the run there.

A few caveats: it's Chat Completions only for now, the reservation is an estimate so set max_tokens, and only calls through the gateway count. Tested with agent-framework-core 1.19.0 and agent-framework-openai 1.14.4.

Setup and the full snippet: recipe. I maintain Inferrail (open source, Apache-2.0).