You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/llms-full.txt
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -38,7 +38,7 @@ The Agents SDK delivers a focused set of Python primitives—agents, tools, guar
38
38
- [Realtime guide](https://openai.github.io/openai-agents-python/realtime/guide/): Deep dive into realtime session lifecycle, structured input, approvals, interruptions, and low-level transport control.
39
39
40
40
## Models and Provider Integrations
41
-
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Covers OpenAI model selection, non-OpenAI provider patterns, websocket transport, and the SDK's best-effort LiteLLM guidance in one place.
41
+
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Covers OpenAI model selection, non-OpenAI provider patterns, websocket transport, and third-party adapter guidance in one place.
42
42
43
43
## API Reference – Agents SDK Core
44
44
- [API index](https://openai.github.io/openai-agents-python/ref/index/): Directory of all documented modules, classes, and functions in the SDK.
@@ -103,7 +103,7 @@ The Agents SDK delivers a focused set of Python primitives—agents, tools, guar
103
103
## API Reference – Extensions
104
104
- [Handoff filters extension](https://openai.github.io/openai-agents-python/ref/extensions/handoff_filters/): Build filters that decide whether to trigger a handoff.
105
105
- [Handoff prompt extension](https://openai.github.io/openai-agents-python/ref/extensions/handoff_prompt/): Customize prompt templates used when transferring control.
106
-
- [LiteLLM extension](https://openai.github.io/openai-agents-python/ref/extensions/litellm/): Adapter for using LiteLLM-managed providers inside the SDK.
106
+
- [Third-party adapters API reference](https://openai.github.io/openai-agents-python/ref/extensions/): API reference entry point for Any-LLM and LiteLLM model adapters and providers.
107
107
- [SQLAlchemy session memory](https://openai.github.io/openai-agents-python/ref/extensions/memory/sqlalchemy_session/): Persist agent session history to SQL databases.
Copy file name to clipboardExpand all lines: docs/models/index.md
+21-11Lines changed: 21 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ Start with the simplest path that fits your setup:
16
16
| Use one non-OpenAI provider | Start with the built-in provider integration points |[Non-OpenAI models](#non-openai-models)|
17
17
| Mix models or providers across agents | Select providers per run or per agent and review feature differences |[Mixing models in one workflow](#mixing-models-in-one-workflow) and [Mixing models across providers](#mixing-models-across-providers)|
18
18
| Tune advanced OpenAI Responses request settings | Use `ModelSettings` on the OpenAI Responses path |[Advanced OpenAI Responses settings](#advanced-openai-responses-settings)|
19
-
| Use LiteLLM for non-OpenAI Chat Completions providers|Treat LiteLLM as a beta fallback |[LiteLLM](#litellm)|
19
+
| Use a third-party adapter for non-OpenAI or mixed-provider routing|Compare the supported beta adapters and validate the provider path you plan to ship |[Third-party adapters](#third-party-adapters)|
20
20
21
21
## OpenAI models
22
22
@@ -135,7 +135,7 @@ result = await Runner.run(
135
135
136
136
#### Advanced routing with `MultiProvider`
137
137
138
-
If you need prefix-based model routing (for example mixing `openai/...` and `litellm/...` model names in one run), use [`MultiProvider`][agents.MultiProvider] and set `openai_use_responses_websocket=True` there instead.
138
+
If you need prefix-based model routing (for example mixing `openai/...` and `any-llm/...` model names in one run), use [`MultiProvider`][agents.MultiProvider] and set `openai_use_responses_websocket=True` there instead.
139
139
140
140
`MultiProvider` keeps two historical defaults:
141
141
@@ -180,7 +180,7 @@ If you use a custom OpenAI-compatible endpoint or proxy, websocket transport als
180
180
181
181
## Non-OpenAI models
182
182
183
-
If you need a non-OpenAI provider, start with the SDK's built-in provider integration points. In many setups, this is enough without adding LiteLLM. Examples for each pattern live in [examples/model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/).
183
+
If you need a non-OpenAI provider, start with the SDK's built-in provider integration points. In many setups, this is enough without adding a third-party adapter. Examples for each pattern live in [examples/model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/).
184
184
185
185
### Ways to integrate non-OpenAI providers
186
186
@@ -189,7 +189,7 @@ If you need a non-OpenAI provider, start with the SDK's built-in provider integr
189
189
|[`set_default_openai_client`][agents.set_default_openai_client]| One OpenAI-compatible endpoint should be the default for most or all agents | Global default |
190
190
|[`ModelProvider`][agents.models.interface.ModelProvider]| One custom provider should apply to a single run | Per run |
191
191
|[`Agent.model`][agents.agent.Agent.model]| Different agents need different providers or concrete model objects | Per agent |
192
-
|LiteLLM (beta)| You need LiteLLM-specific provider coverage or routing | See [LiteLLM](#litellm)|
192
+
|Third-party adapter| You need adapter-managed provider coverage or routing that the built-in paths do not provide | See [Third-party adapters](#third-party-adapters)|
193
193
194
194
You can integrate other LLM providers with these built-in paths:
195
195
@@ -404,7 +404,7 @@ Stateful follow-up requests using `previous_response_id` or `conversation_id` ar
404
404
- An agent can override only part of `retry.backoff` and keep sibling backoff fields from the runner.
405
405
-`policy` is runtime-only, so serialized `ModelSettings` keep `max_retries` and `backoff` but omit the callback itself.
406
406
407
-
For fuller examples, see [`examples/basic/retry.py`](https://github.com/openai/openai-agents-python/tree/main/examples/basic/retry.py) and [`examples/basic/retry_litellm.py`](https://github.com/openai/openai-agents-python/tree/main/examples/basic/retry_litellm.py).
407
+
For fuller examples, see [`examples/basic/retry.py`](https://github.com/openai/openai-agents-python/tree/main/examples/basic/retry.py) and the [adapter-backed retry example](https://github.com/openai/openai-agents-python/tree/main/examples/basic/retry_litellm.py).
408
408
409
409
## Troubleshooting non-OpenAI providers
410
410
@@ -443,14 +443,24 @@ You need to be aware of feature differences between model providers, or you may
443
443
- Filter out multimodal inputs before calling models that are text-only
444
444
- Be aware that providers that don't support structured JSON outputs will occasionally produce invalid JSON.
445
445
446
-
## LiteLLM
446
+
## Third-party adapters
447
447
448
-
LiteLLM support is included on a best-effort, beta basis for cases where you need to bring non-OpenAI providers into an Agents SDK workflow.
448
+
Reach for a third-party adapter only when the SDK's built-in provider integration points are not enough. If you are using OpenAI models only with this SDK, prefer the built-in [`OpenAIResponsesModel`][agents.models.openai_responses.OpenAIResponsesModel] path instead of Any-LLM or LiteLLM. Third-party adapters are for cases where you need to combine OpenAI models with non-OpenAI providers, or need adapter-managed provider coverage or routing that the built-in paths do not provide. Adapters add another compatibility layer between the SDK and the upstream model provider, so feature support and request semantics can vary by provider. The SDK currently includes Any-LLM and LiteLLM as best-effort, beta adapter integrations.
449
449
450
-
If you are using OpenAI models with this SDK, we recommend the built-in [`OpenAIResponsesModel`][agents.models.openai_responses.OpenAIResponsesModel] path instead of LiteLLM.
450
+
### Any-LLM
451
451
452
-
If you need to combine OpenAI models with non-OpenAI providers, especially through Chat Completions-compatible APIs, LiteLLM is available as a beta option, but it may not be the optimal choice for every setup.
452
+
Any-LLM support is included on a best-effort, beta basis for cases where you need Any-LLM-managed provider coverage or routing.
453
453
454
-
If you need LiteLLM for a non-OpenAI provider, install `openai-agents[litellm]`, then start from [`examples/model_providers/litellm_auto.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/litellm_auto.py) or [`examples/model_providers/litellm_provider.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/litellm_provider.py). You can either use `litellm/...` model names or instantiate [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel] directly.
454
+
Depending on the upstream provider path, Any-LLM may use the Responses API, Chat Completions-compatible APIs, or provider-specific compatibility layers.
455
455
456
-
If you want LiteLLM responses to populate the SDK's usage metrics, pass `ModelSettings(include_usage=True)`.
456
+
If you need Any-LLM, install `openai-agents[any-llm]`, then start from [`examples/model_providers/any_llm_auto.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/any_llm_auto.py) or [`examples/model_providers/any_llm_provider.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/any_llm_provider.py). You can use `any-llm/...` model names with [`MultiProvider`][agents.MultiProvider], instantiate `AnyLLMModel` directly, or use `AnyLLMProvider` at run scope. If you need to pin the model surface explicitly, pass `api="responses"` or `api="chat_completions"` when constructing `AnyLLMModel`.
457
+
458
+
Any-LLM remains a third-party adapter layer, so provider dependencies and capability gaps are defined upstream by Any-LLM rather than by the SDK. Usage metrics are propagated automatically when the upstream provider returns them, but streamed Chat Completions backends may require `ModelSettings(include_usage=True)` before they emit usage chunks. Validate the exact provider backend you plan to deploy if you depend on structured outputs, tool calling, usage reporting, or Responses-specific behavior.
459
+
460
+
### LiteLLM
461
+
462
+
LiteLLM support is included on a best-effort, beta basis for cases where you need LiteLLM-specific provider coverage or routing.
463
+
464
+
If you need LiteLLM, install `openai-agents[litellm]`, then start from [`examples/model_providers/litellm_auto.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/litellm_auto.py) or [`examples/model_providers/litellm_provider.py`](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers/litellm_provider.py). You can use `litellm/...` model names or instantiate [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel] directly.
465
+
466
+
Some LiteLLM-backed providers do not populate SDK usage metrics by default. If you need usage reporting, pass `ModelSettings(include_usage=True)` and validate the exact provider backend you plan to deploy if you depend on structured outputs, tool calling, usage reporting, or adapter-specific routing behavior.
Copy file name to clipboardExpand all lines: docs/tracing.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -103,18 +103,18 @@ To customize this default setup, to send traces to alternative or additional bac
103
103
104
104
## Tracing with non-OpenAI models
105
105
106
-
You can use an OpenAI API key with non-OpenAI Models to enable free tracing in the OpenAI Traces dashboard without needing to disable tracing.
106
+
You can use an OpenAI API key with non-OpenAI models to enable free tracing in the OpenAI Traces dashboard without needing to disable tracing. See the [Third-party adapters](models/index.md#third-party-adapters) section in the Models guide for adapter selection and setup caveats.
107
107
108
108
```python
109
109
import os
110
110
from agents import set_tracing_export_api_key, Agent, Runner
111
-
from agents.extensions.models.litellm_modelimportLitellmModel
111
+
from agents.extensions.models.any_llm_modelimportAnyLLMModel
Usage is aggregated across all model calls during the run (including tool calls and handoffs).
31
31
32
-
### Enabling usage with LiteLLM models
32
+
### Enabling usage with third-party adapters
33
33
34
-
LiteLLM providers do not report usage metrics by default. When you are using [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel], pass `ModelSettings(include_usage=True)` to your agent so that LiteLLM responses populate `result.context_wrapper.usage`. See the [LiteLLM note](models/index.md#litellm) in the Models guide for setup guidance and examples.
34
+
Usage reporting varies across third-party adapters and provider backends. If you rely on adapter-backed models and need accurate `result.context_wrapper.usage` values:
35
35
36
-
```python
37
-
from agents import Agent, ModelSettings, Runner
38
-
from agents.extensions.models.litellm_model import LitellmModel
- With `AnyLLMModel`, usage is propagated automatically when the upstream provider returns it. For streamed Chat Completions backends, you may need `ModelSettings(include_usage=True)` before usage chunks are emitted.
37
+
- With `LitellmModel`, some provider backends do not report usage by default, so `ModelSettings(include_usage=True)` is often required.
45
38
46
-
result =await Runner.run(agent, "What's the weather in Tokyo?")
47
-
print(result.context_wrapper.usage.total_tokens)
48
-
```
39
+
Review the adapter-specific notes in the [Third-party adapters](models/index.md#third-party-adapters) section of the Models guide and validate the exact provider backend you plan to deploy.
0 commit comments