Skip to content

Python: Add timeout parameter to OpenAI client classes - #9924

Draft
Isham Mohamed (IshamMohamed) wants to merge 4 commits into
microsoft:mainfrom
IshamMohamed:main
Draft

Isham Mohamed (IshamMohamed) wants to merge 4 commits into
microsoft:mainfrom
IshamMohamed:main

Conversation

@IshamMohamed

@IshamMohamed Isham Mohamed (IshamMohamed) commented Nov 1, 2026 •

Copy link
Copy Markdown

(OpenAIChatClient, OpenAIChatCompletionClient, and OpenAIEmbeddingClient) for consistent HTTP timeout control across all OpenAI integrations.

Motivation & Context

  1. Why is this change required?
    This change is required because the high-level OpenAI client classes (OpenAIChatClient, OpenAIChatCompletionClient, OpenAIEmbeddingClient) did not expose a timeout parameter, while the underlying raw clients (RawOpenAIChatClient, RawOpenAIChatCompletionClient, RawOpenAIEmbeddingClient) already supported it. This prevented developers from controlling HTTP timeouts when using the framework's public client APIs.

  2. What problem does it solve?
    It solves the problem of developers being unable to override the default OpenAI SDK timeout (connect: 5s, read/write/pool: 600s) through the framework's high-level clients. Previously, developers had to manually configure async_client instances to control timeouts, which was inconsistent with the raw client API.

  3. What scenario does it contribute to?
    It contributes to the scenario where applications need different timeout behaviors for OpenAI API requests (chat, chat completions, embeddings) without having to manually configure async_client instances. For example, a long-running embedding request might need a 120s timeout while chat requests use 30s.

  4. If it fixes an open issue, please link to the issue below.
    Fixes Python: [Feature]: Make timeout configurable on framework OpenAI clients (chat, chat completion, embedding) #9923

  5. Scope of work is only limited to timeout on the OpenAI client classes because it depends on RawOpenAI*Client classes and they have timeout exposed. A seperate PR may be requirered to implement max_retries (similar to Python: [Feature]: Add max_retries (or OpenAI client kwargs) to FoundryChatClient #9719) because that work requires RawOpenAI*Client too to have max_retries exposed.

Description & Review Guide

The overall approach follows the existing pattern already established in the raw OpenAI client classes (RawOpenAIChatClient, RawOpenAIChatCompletionClient, RawOpenAIEmbeddingClient), which already supported a timeout parameter. The implementation uses the load_openai_service_settings() infrastructure to handle both OpenAI and Azure routing consistently.

The underlying design ensures a single timeout value applies to all request phases (connect, read, write, pool). When async_client is provided, the timeout parameter is ignored, allowing users to configure timeout directly on their pre-configured client. Default behavior is preserved (None = OpenAI SDK defaults: connect 5s, read/write/pool 600s).

  • What are the major changes?
    Added timeout: float | None = None to all three __init__ overloads of OpenAIChatClient, OpenAIChatCompletionClient, and OpenAIEmbeddingClient. Updated docstrings to document the timeout parameter. Forwarded timeout through load_openai_service_settings() for proper handling.
  • What is the impact of these changes?
    Developers can now pass timeout to all three client classes. Default behavior is preserved (None = SDK default). Timeout is ignored when async_client is provided. (PS: not added to NvidiaEmbeddingClient or FoundryLocalClient as they manage their own clients). The focust is only on directly OpenAI related clients on the agent-framework.
  • *What Reviewers Should Focus On
    • Verify timeout is added to all three overloads for each client class
    • Confirm docstrings describe the parameter correctly
    • Ensure timeout=timeout is passed to load_openai_service_settings()
    • Check backward compatibility (default None preserves SDK defaults)
    • Confirm tests cover the new parameter

Related Issue

Fixes #9923

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details:

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

(OpenAIChatClient, OpenAIChatCompletionClient, and
OpenAIEmbeddingClient) for consistent HTTP timeout control across all
OpenAI integrations.
Copilot AI balanced review requested due to automatic review settings November 1, 2026 00:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A constructor indentation error prevents Chat Completions imports, while missing public timeout support and superclass forwarding leave the new timeout behavior incomplete.

4 open issues
What changed in this PR

This PR aims to address #9923 by exposing HTTP timeout control through Agent Framework’s public Python OpenAI clients, without requiring a custom SDK client.

Changes:

  • Adds timeout parameters to Responses and embedding client constructors, and the raw Chat Completions constructor.
  • Forwards embedding and raw Chat Completions timeouts through shared settings.
  • Documents timeout defaults and behavior when an SDK client is supplied.
File Description
python/​packages/​openai/​agent_framework_openai/​_embedding_client.py Adds timeout parameters, documentation, and forwarding for raw and public embedding clients.
python/​packages/​openai/​agent_framework_openai/​_chat_completion_client.py Adds timeout support to the raw Chat Completions constructor.
python/​packages/​openai/​agent_framework_openai/​_chat_client.py Adds timeout parameters and documentation to the public Responses client.

Review effort: Balanced

…ation to restore imports'

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

This branch was previously deployed

1 inactive deployment
github-app-auth — 1bf5688f Deployed Nov 1, 2026 by IshamMohamed via add_label #26775
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Feature]: Make timeout configurable on framework OpenAI clients (chat, chat completion, embedding)

2 participants