[Beta Sessions / Managed Agents] MCP tool-result image content blocks reach the model as base64 text instead of vision

[arcaputo3]

Summary

In the Beta Managed Agents / Sessions API (self-hosted environment — Anthropic runs the agent loop; tools execute in our sandbox), when a configured remote MCP server returns a tool result that contains an image content block ({"type":"image","data":"<base64>","mimeType":"image/jpeg"}), the image is not delivered to the model as vision. The base64 payload instead reaches the model as raw text inside the tool result.

Two consequences:

1. The model cannot see the image — it can only work from any accompanying text/OCR block, and explicitly reports it "cannot see" the picture.
2. Large token waste — the base64 string is tokenized as text (tens of thousands of tokens per image), with no benefit.

Expected

MCP tool-result image content blocks should be forwarded to the model as image (vision) input — the same way they are in a normal Claude session.

Actual

The model receives the base64 as text.

Isolation already done

• The same MCP server, called from a normal Claude session (claude.ai / Claude Code via an MCP connector), returns images that render correctly and the model can see them → the server emits valid ImageContent blocks.
• For Managed Agents, MCP tools execute server-side: the Python SDK's session runner explicitly does not dispatch agent.mcp_tool_use (in-code comment: "MCP tools run server-side and the runner never sees a result to post for them"). So this is in the managed-agents MCP relay, not the local tool runtime.
• BetaManagedAgentsAgentMcpToolResultEvent's content union already includes BetaManagedAgentsImageBlock, so the type system supports image results — the relay appears to flatten ImageContent into a text/base64 representation instead of forwarding it as an image block to the model.

Environment

• anthropic (Python) 0.109.1
• Beta Managed Agents / Sessions, self-hosted environment

Related

Related to but distinct from #1329: that issue is about the claude.ai UI not rendering tool images inline to the user (where "Claude can see the image"). This report is the opposite — in Managed Agents the model itself does not receive the image as vision.

Repro

Configure a managed agent with a remote MCP server whose tool returns [TextContent, ImageContent]; run a session that calls that tool; observe that the model receives the base64 as text rather than seeing the image. Happy to provide a minimal MCP server and a session id on request.

[arcaputo3]

Confirmed against a live managed-agents (Beta sessions) session. The agent used a remote MCP server whose search/fetch tools return [TextContent, ImageContent]:

• 4 agent.mcp_tool_result events were delivered to the model.
• Each carried ~270,000–496,000 characters of base64 image data inline as TEXT (~1.5M characters total across the four — hundreds of thousands of tokens of unusable base64).
• Zero image content blocks reached the model — none of the agent.mcp_tool_result contents had an image-type block.

So the managed-agents relay is flattening the MCP ImageContent into base64 text rather than forwarding it as vision. Net effect: the model can't see the images and pays a large token cost for the base64 text. The same MCP server's images render correctly (as vision) in a normal Claude session, so the server is emitting valid ImageContent.

Happy to share a session id and a minimal reproducing MCP server privately via support.