Skip to content

Commit 13f4a74

Browse files
authored
docs: update pages to add any-llm adapter (#2715)
1 parent 0a5b8c9 commit 13f4a74

File tree

9 files changed

+50
-39
lines changed

9 files changed

+50
-39
lines changed

docs/examples.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Check out a variety of sample implementations of the SDK in the examples section
2929
- File handling (local and remote, images and PDFs)
3030
- Usage tracking
3131
- Runner-managed retry settings (`examples/basic/retry.py`)
32-
- Runner-managed retries with LiteLLM (`examples/basic/retry_litellm.py`)
32+
- Runner-managed retries through a third-party adapter (`examples/basic/retry_litellm.py`)
3333
- Non-strict output types
3434
- Previous response ID usage
3535

@@ -68,7 +68,7 @@ Check out a variety of sample implementations of the SDK in the examples section
6868
- Stateless Responses compaction with `ModelSettings(store=False)` (`examples/memory/compaction_session_stateless_example.py`)
6969

7070
- **[model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers):**
71-
Explore how to use non-OpenAI models with the SDK, including custom providers and LiteLLM integration.
71+
Explore how to use non-OpenAI models with the SDK, including custom providers and third-party adapters.
7272

7373
- **[realtime](https://github.com/openai/openai-agents-python/tree/main/examples/realtime):**
7474
Examples showing how to build real-time experiences using the SDK, including:

docs/llms-full.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ The Agents SDK delivers a focused set of Python primitives—agents, tools, guar
3838
- [Realtime guide](https://openai.github.io/openai-agents-python/realtime/guide/): Deep dive into realtime session lifecycle, structured input, approvals, interruptions, and low-level transport control.
3939

4040
## Models and Provider Integrations
41-
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Covers OpenAI model selection, non-OpenAI provider patterns, websocket transport, and the SDK's best-effort LiteLLM guidance in one place.
41+
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Covers OpenAI model selection, non-OpenAI provider patterns, websocket transport, and third-party adapter guidance in one place.
4242

4343
## API Reference – Agents SDK Core
4444
- [API index](https://openai.github.io/openai-agents-python/ref/index/): Directory of all documented modules, classes, and functions in the SDK.
@@ -103,7 +103,7 @@ The Agents SDK delivers a focused set of Python primitives—agents, tools, guar
103103
## API Reference – Extensions
104104
- [Handoff filters extension](https://openai.github.io/openai-agents-python/ref/extensions/handoff_filters/): Build filters that decide whether to trigger a handoff.
105105
- [Handoff prompt extension](https://openai.github.io/openai-agents-python/ref/extensions/handoff_prompt/): Customize prompt templates used when transferring control.
106-
- [LiteLLM extension](https://openai.github.io/openai-agents-python/ref/extensions/litellm/): Adapter for using LiteLLM-managed providers inside the SDK.
106+
- [Third-party adapters API reference](https://openai.github.io/openai-agents-python/ref/extensions/): API reference entry point for Any-LLM and LiteLLM model adapters and providers.
107107
- [SQLAlchemy session memory](https://openai.github.io/openai-agents-python/ref/extensions/memory/sqlalchemy_session/): Persist agent session history to SQL databases.
108108

109109
## Optional

docs/llms.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,10 +49,10 @@ The SDK focuses on a concise set of primitives so you can orchestrate multi-agen
4949
- [Tracing APIs](https://openai.github.io/openai-agents-python/ref/tracing/index/): Programmatic interfaces for creating traces, spans, and integrating custom processors.
5050
- [Realtime APIs](https://openai.github.io/openai-agents-python/ref/realtime/agent/): Classes for realtime agents, runners, sessions, and event payloads.
5151
- [Voice APIs](https://openai.github.io/openai-agents-python/ref/voice/pipeline/): Configure voice pipelines, inputs, events, and model adapters.
52-
- [Extensions](https://openai.github.io/openai-agents-python/ref/extensions/handoff_filters/): Extend the SDK with custom handoff filters, prompts, LiteLLM integration, and SQLAlchemy session memory.
52+
- [Extensions](https://openai.github.io/openai-agents-python/ref/extensions/handoff_filters/): Extend the SDK with custom handoff filters, prompts, third-party adapters, and SQLAlchemy session memory.
5353

5454
## Models and Providers
55-
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Overview of OpenAI models, non-OpenAI provider patterns, websocket transport, and the SDK's best-effort LiteLLM guidance.
55+
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Overview of OpenAI models, non-OpenAI provider patterns, websocket transport, and third-party adapter guidance.
5656

5757
## Optional
5858
- [Release notes](https://openai.github.io/openai-agents-python/release/): Track SDK changes, migration notes, and deprecations.

docs/models/index.md

Lines changed: 21 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Start with the simplest path that fits your setup:
1616
| Use one non-OpenAI provider | Start with the built-in provider integration points | [Non-OpenAI models](#non-openai-models) |
1717
| Mix models or providers across agents | Select providers per run or per agent and review feature differences | [Mixing models in one workflow](#mixing-models-in-one-workflow) and [Mixing models across providers](#mixing-models-across-providers) |
1818
| Tune advanced OpenAI Responses request settings | Use `ModelSettings` on the OpenAI Responses path | [Advanced OpenAI Responses settings](#advanced-openai-responses-settings) |
19-
| Use LiteLLM for non-OpenAI Chat Completions providers | Treat LiteLLM as a beta fallback | [LiteLLM](#litellm) |
19+
| Use a third-party adapter for non-OpenAI or mixed-provider routing | Compare the supported beta adapters and validate the provider path you plan to ship | [Third-party adapters](#third-party-adapters) |
2020

2121
## OpenAI models
2222

@@ -135,7 +135,7 @@ result = await Runner.run(
135135

136136
#### Advanced routing with `MultiProvider`
137137

138-
If you need prefix-based model routing (for example mixing `openai/...` and `litellm/...` model names in one run), use [`MultiProvider`][agents.MultiProvider] and set `openai_use_responses_websocket=True` there instead.
138+
If you need prefix-based model routing (for example mixing `openai/...` and `any-llm/...` model names in one run), use [`MultiProvider`][agents.MultiProvider] and set `openai_use_responses_websocket=True` there instead.
139139

140140
`MultiProvider` keeps two historical defaults:
141141

@@ -180,7 +180,7 @@ If you use a custom OpenAI-compatible endpoint or proxy, websocket transport als
180180

181181
## Non-OpenAI models
182182

183-
If you need a non-OpenAI provider, start with the SDK's built-in provider integration points. In many setups, this is enough without adding LiteLLM. Examples for each pattern live in [examples/model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/).
183+
If you need a non-OpenAI provider, start with the SDK's built-in provider integration points. In many setups, this is enough without adding a third-party adapter. Examples for each pattern live in [examples/model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/).
184184

185185
### Ways to integrate non-OpenAI providers
186186

@@ -189,7 +189,7 @@ If you need a non-OpenAI provider, start with the SDK's built-in provider integr
189189
| [`set_default_openai_client`][agents.set_default_openai_client] | One OpenAI-compatible endpoint should be the default for most or all agents | Global default |
190190
| [`ModelProvider`][agents.models.interface.ModelProvider] | One custom provider should apply to a single run | Per run |
191191
| [`Agent.model`][agents.agent.Agent.model] | Different agents need different providers or concrete model objects | Per agent |
192-
| LiteLLM (beta) | You need LiteLLM-specific provider coverage or routing | See [LiteLLM](#litellm) |
192+
| Third-party adapter | You need adapter-managed provider coverage or routing that the built-in paths do not provide | See [Third-party adapters](#third-party-adapters) |
193193

194194
You can integrate other LLM providers with these built-in paths:
195195

@@ -404,7 +404,7 @@ Stateful follow-up requests using `previous_response_id` or `conversation_id` ar
404404
- An agent can override only part of `retry.backoff` and keep sibling backoff fields from the runner.
405405
- `policy` is runtime-only, so serialized `ModelSettings` keep `max_retries` and `backoff` but omit the callback itself.
406406

407-
For fuller examples, see [`examples/basic/retry.py`](https://github.com/openai/openai-agents-python/tree/main/examples/basic/retry.py) and [`examples/basic/retry_litellm.py`](https://github.com/openai/openai-agents-python/tree/main/examples/basic/retry_litellm.py).
407+
For fuller examples, see [`examples/basic/retry.py`](https://github.com/openai/openai-agents-python/tree/main/examples/basic/retry.py) and the [adapter-backed retry example](https://github.com/openai/openai-agents-python/tree/main/examples/basic/retry_litellm.py).
408408

409409
## Troubleshooting non-OpenAI providers
410410

@@ -443,14 +443,24 @@ You need to be aware of feature differences between model providers, or you may
443443
- Filter out multimodal inputs before calling models that are text-only
444444
- Be aware that providers that don't support structured JSON outputs will occasionally produce invalid JSON.
445445

446-
## LiteLLM
446+
## Third-party adapters
447447

448-
LiteLLM support is included on a best-effort, beta basis for cases where you need to bring non-OpenAI providers into an Agents SDK workflow.
448+
Reach for a third-party adapter only when the SDK's built-in provider integration points are not enough. If you are using OpenAI models only with this SDK, prefer the built-in [`OpenAIResponsesModel`][agents.models.openai_responses.OpenAIResponsesModel] path instead of Any-LLM or LiteLLM. Third-party adapters are for cases where you need to combine OpenAI models with non-OpenAI providers, or need adapter-managed provider coverage or routing that the built-in paths do not provide. Adapters add another compatibility layer between the SDK and the upstream model provider, so feature support and request semantics can vary by provider. The SDK currently includes Any-LLM and LiteLLM as best-effort, beta adapter integrations.
449449

450-
If you are using OpenAI models with this SDK, we recommend the built-in [`OpenAIResponsesModel`][agents.models.openai_responses.OpenAIResponsesModel] path instead of LiteLLM.
450+
### Any-LLM
451451

452-
If you need to combine OpenAI models with non-OpenAI providers, especially through Chat Completions-compatible APIs, LiteLLM is available as a beta option, but it may not be the optimal choice for every setup.
452+
Any-LLM support is included on a best-effort, beta basis for cases where you need Any-LLM-managed provider coverage or routing.
453453

454-
If you need LiteLLM for a non-OpenAI provider, install `openai-agents[litellm]`, then start from [`examples/model_providers/litellm_auto.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/litellm_auto.py) or [`examples/model_providers/litellm_provider.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/litellm_provider.py). You can either use `litellm/...` model names or instantiate [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel] directly.
454+
Depending on the upstream provider path, Any-LLM may use the Responses API, Chat Completions-compatible APIs, or provider-specific compatibility layers.
455455

456-
If you want LiteLLM responses to populate the SDK's usage metrics, pass `ModelSettings(include_usage=True)`.
456+
If you need Any-LLM, install `openai-agents[any-llm]`, then start from [`examples/model_providers/any_llm_auto.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/any_llm_auto.py) or [`examples/model_providers/any_llm_provider.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/any_llm_provider.py). You can use `any-llm/...` model names with [`MultiProvider`][agents.MultiProvider], instantiate `AnyLLMModel` directly, or use `AnyLLMProvider` at run scope. If you need to pin the model surface explicitly, pass `api="responses"` or `api="chat_completions"` when constructing `AnyLLMModel`.
457+
458+
Any-LLM remains a third-party adapter layer, so provider dependencies and capability gaps are defined upstream by Any-LLM rather than by the SDK. Usage metrics are propagated automatically when the upstream provider returns them, but streamed Chat Completions backends may require `ModelSettings(include_usage=True)` before they emit usage chunks. Validate the exact provider backend you plan to deploy if you depend on structured outputs, tool calling, usage reporting, or Responses-specific behavior.
459+
460+
### LiteLLM
461+
462+
LiteLLM support is included on a best-effort, beta basis for cases where you need LiteLLM-specific provider coverage or routing.
463+
464+
If you need LiteLLM, install `openai-agents[litellm]`, then start from [`examples/model_providers/litellm_auto.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/litellm_auto.py) or [`examples/model_providers/litellm_provider.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/litellm_provider.py). You can use `litellm/...` model names or instantiate [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel] directly.
465+
466+
Some LiteLLM-backed providers do not populate SDK usage metrics by default. If you need usage reporting, pass `ModelSettings(include_usage=True)` and validate the exact provider backend you plan to deploy if you depend on structured outputs, tool calling, usage reporting, or adapter-specific routing behavior.

docs/models/litellm.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# LiteLLM
22

33
<script>
4-
window.location.replace("../#litellm");
4+
window.location.replace("../#third-party-adapters");
55
</script>
66

7-
This page moved to the [LiteLLM section in Models](index.md#litellm).
7+
This page moved to the [Third-party adapters section in Models](index.md#third-party-adapters).
88

99
If you are not redirected automatically, use the link above.

docs/ref/extensions/litellm.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
11
# `LiteLLM Models`
22

3-
::: agents.extensions.models.litellm_model
3+
<script>
4+
window.location.replace("../third_party_adapters/");
5+
</script>
6+
7+
This page moved to the [Third-party adapters API reference](third_party_adapters.md).
8+
9+
If you are not redirected automatically, use the link above.

docs/tracing.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -103,18 +103,18 @@ To customize this default setup, to send traces to alternative or additional bac
103103

104104
## Tracing with non-OpenAI models
105105

106-
You can use an OpenAI API key with non-OpenAI Models to enable free tracing in the OpenAI Traces dashboard without needing to disable tracing.
106+
You can use an OpenAI API key with non-OpenAI models to enable free tracing in the OpenAI Traces dashboard without needing to disable tracing. See the [Third-party adapters](models/index.md#third-party-adapters) section in the Models guide for adapter selection and setup caveats.
107107

108108
```python
109109
import os
110110
from agents import set_tracing_export_api_key, Agent, Runner
111-
from agents.extensions.models.litellm_model import LitellmModel
111+
from agents.extensions.models.any_llm_model import AnyLLMModel
112112

113113
tracing_api_key = os.environ["OPENAI_API_KEY"]
114114
set_tracing_export_api_key(tracing_api_key)
115115

116-
model = LitellmModel(
117-
model="your-model-name",
116+
model = AnyLLMModel(
117+
model="your-provider/your-model-name",
118118
api_key="your-api-key",
119119
)
120120

docs/usage.md

Lines changed: 5 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -29,23 +29,14 @@ print("Total tokens:", usage.total_tokens)
2929

3030
Usage is aggregated across all model calls during the run (including tool calls and handoffs).
3131

32-
### Enabling usage with LiteLLM models
32+
### Enabling usage with third-party adapters
3333

34-
LiteLLM providers do not report usage metrics by default. When you are using [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel], pass `ModelSettings(include_usage=True)` to your agent so that LiteLLM responses populate `result.context_wrapper.usage`. See the [LiteLLM note](models/index.md#litellm) in the Models guide for setup guidance and examples.
34+
Usage reporting varies across third-party adapters and provider backends. If you rely on adapter-backed models and need accurate `result.context_wrapper.usage` values:
3535

36-
```python
37-
from agents import Agent, ModelSettings, Runner
38-
from agents.extensions.models.litellm_model import LitellmModel
39-
40-
agent = Agent(
41-
name="Assistant",
42-
model=LitellmModel(model="your/model", api_key="..."),
43-
model_settings=ModelSettings(include_usage=True),
44-
)
36+
- With `AnyLLMModel`, usage is propagated automatically when the upstream provider returns it. For streamed Chat Completions backends, you may need `ModelSettings(include_usage=True)` before usage chunks are emitted.
37+
- With `LitellmModel`, some provider backends do not report usage by default, so `ModelSettings(include_usage=True)` is often required.
4538

46-
result = await Runner.run(agent, "What's the weather in Tokyo?")
47-
print(result.context_wrapper.usage.total_tokens)
48-
```
39+
Review the adapter-specific notes in the [Third-party adapters](models/index.md#third-party-adapters) section of the Models guide and validate the exact provider backend you plan to deploy.
4940

5041
## Per-request usage tracking
5142

mkdocs.yml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -156,7 +156,11 @@ plugins:
156156
- Extensions:
157157
- ref/extensions/handoff_filters.md
158158
- ref/extensions/handoff_prompt.md
159-
- ref/extensions/litellm.md
159+
- Third-party adapters:
160+
- Any-LLM model: ref/extensions/models/any_llm_model.md
161+
- Any-LLM provider: ref/extensions/models/any_llm_provider.md
162+
- LiteLLM model: ref/extensions/models/litellm_model.md
163+
- LiteLLM provider: ref/extensions/models/litellm_provider.md
160164
- ref/extensions/tool_output_trimmer.md
161165
- ref/extensions/memory/sqlalchemy_session.md
162166
- ref/extensions/memory/async_sqlite_session.md

0 commit comments

Comments
 (0)