Skip to content

Python: Preserve request metadata in hosted responses - #9745

Open
Aman (its-amann) wants to merge 4 commits into
microsoft:mainfrom
its-amann:fix/foundry-response-metadata-9529
Open

Aman (its-amann) wants to merge 4 commits into
microsoft:mainfrom
its-amann:fix/foundry-response-metadata-9529

Conversation

@its-amann

@its-amann Aman (its-amann) commented Oct 27, 2026 •

Copy link
Copy Markdown

Motivation & Context

Requests to ResponsesHostServer can include metadata to label or correlate a turn, but fresh responses do not retain it in their events or stored response. Clients reading the response back cannot recover the values they sent.

Description & Review Guide

  • What are the major changes? Seed fresh response streams from sanitized request metadata, restore recovery streams from trusted snapshots, and persist host-owned recovery values in the snapshot's private envelope. Existing snapshots using the SDK metadata bag remain readable.
  • What is the impact of these changes? All 16 public metadata entries survive response events and stored responses. Provider recovery resumes at the cursor saved with the output, preserving usage and incomplete reasons without replaying completed batches. Private state is removed at the SDK's HTTP and SSE boundaries.
  • What do you want reviewers to focus on? Checkpoint/output consistency across crashes, legacy recovery compatibility, and the fresh-request trust boundary. Regressions cover 15/16 public keys, crashes before/after a checkpoint, repeated recovery, and completed/truncated responses.

Related Issue

Fixes #9529

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: Issue analysis, implementation, regression tests and this PR description were assisted.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after a language prefix) — a workflow keeps the label and the title prefix in sync automatically.

Copilot AI balanced review requested due to automatic review settings October 27, 2026 07:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Oct 27, 2026
@eavanvalkenburg

Copy link
Copy Markdown
Member

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): 6037edd0fa65
Model: gpt-5.6-sol

Overview

The change correctly separates fresh-request seeding from persisted-response recovery and adds broad round-trip coverage for omitted, empty, and populated metadata. The SDK also deep-copies request metadata, and the new tests cover streaming, non-streaming, storage, and recovery precedence. Two residual Medium risks remain because public metadata and the SDK's reserved internal metadata share the same map: callers can inject trusted recovery state, and valid maximum-size metadata can prevent the host from recording an incomplete finish reason.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py

Comment thread python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py Outdated
Comment thread python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py Outdated
Drop a client-supplied _internal_metadata bag when a fresh response is
seeded, and keep that bag only for a trusted persisted snapshot. When
public metadata already holds 16 entries, still record a truncating
finish reason so the response ends incomplete instead of failed.
@eavanvalkenburg

Copy link
Copy Markdown
Member

Aman (@its-amann) One material boundary case remains in python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py:349. A caller may supply the supported maximum of 16 public metadata keys, but a later length or content_filter finish needs an additional internal _incomplete_reason slot. The SDK then raises cannot add internal_metadata: response metadata already has 16 keys, converting an expected incomplete response into failed and dropping its output in both streaming and non-streaming modes. Please resolve the public/private metadata-capacity conflict without reducing the 16-key public contract, and add 16-key truncation regressions for both response modes.

A full public metadata map still ends length and content_filter turns
incomplete, with the assistant text and all 16 public keys kept, for
both streaming and non-streaming responses.
@cursor
cursor Bot temporarily deployed to github-app-auth October 27, 2026 10:24 Inactive
@its-amann

Copy link
Copy Markdown
Author

Eduard van Valkenburg (@eavanvalkenburg) thanks, pushed a fix for both findings incl the 16 key case with tests for both modes. ready for review now

This branch was successfully deployed

1 active deployment
github-app-auth — 2e81d252 Deployed Oct 29, 2026 by its-amann via add_label #26549
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: foundry-hosting drops request metadata from the stored response — _create_response_event_stream never passes request to ResponseEventStream

3 participants