Skip to content

feat(mcp): MCP conformance test suites + two OAuth client fixes#1722

Merged
mattzcarey merged 3 commits into
mainfrom
mcp-client-conformance
Jun 17, 2026
Merged

feat(mcp): MCP conformance test suites + two OAuth client fixes#1722
mattzcarey merged 3 commits into
mainfrom
mcp-client-conformance

Conversation

@mattzcarey

Copy link
Copy Markdown
Contributor

🤖 Coauthored with Fable 5 — as a test to get some stuff done.

Closes #1721

What

Adds the official MCP conformance suite (@modelcontextprotocol/conformance, the same harness the MCP TypeScript SDK runs in CI) to this repo:

Suite Implementation under test Entry point
client Agent + MCPClientManager (+ DurableObjectOAuthClientProvider for OAuth) ConformanceHost in conformance/worker.ts
server McpAgent /mcp-agent
server createMcpHandler + WorkerTransport inside an Agent /mcp-handler

Everything under test runs inside workerd (wrangler dev), not a Node approximation — Durable Object storage, the real OAuth callback route, real transports. For the client suite, the conformance CLI spawns a thin Node driver per scenario; the driver forwards the scenario to a fresh agent instance and plays the user's browser for OAuth (follows the authorization redirect into the worker's real /callback route). Both test servers register the same "everything server" feature set ported from the TypeScript SDK's conformance server.

Bugs found and fixed

The client suite immediately caught two real OAuth bugs in MCPClientConnection, fixed in this PR:

  1. Token exchange hit the wrong endpoint (auth/metadata-var3): finishAuth ran on a freshly created transport, losing the resource metadata URL captured from the WWW-Authenticate header on the 401. Discovery fell back to default endpoints, so the code was exchanged at /token even when the authorization server advertised a different location. Fixed by finishing auth on the transport that received the 401.
  2. Permanent failure after mid-session re-auth (auth/scope-step-up): after a mid-session 401 (scope step-up, token revocation) and successful re-authorization, init() failed forever with Already connected to a transport because the previous transport was never detached. Fixed by detaching it before reconnecting.

Both ship with a changeset (patch).

Results

  • Client: 334 checks passing across 26 scenarios, including the full OAuth matrix (metadata variants, scope selection/step-up/retry-limit, token endpoint auth methods, pre-registration, 2025-03-26 backcompat, offline-access drafts). Baselined: SEP-1034 elicitation defaults (elicitation is platform-delegated), CIMD (SHOULD-level warning), client-credentials/cross-app extensions (grant not supported by DurableObjectOAuthClientProvider; cross-app is baselined by the TypeScript SDK too).
  • Servers: 40 checks passing per variant (tools/resources/prompts/logging/completions, sampling, elicitation incl. SEP-1034/SEP-1330, SSE polling + multi-stream). Baselined: json-schema-2020-12 (pinned SDK's zod→JSON Schema conversion doesn't preserve $schema/$defs/additionalProperties verbatim) and dns-rebinding-protection (localhost Host/Origin validation is a local-server concern).

Baselines are documented per-gap in conformance/baseline-*.yml — CI stays green while regressions and new failures are still caught.

CI

New conformance.yml workflow: one client job + a server matrix (mcp-agent, handler), triggered on changes to packages/agents/**.

Running locally

cd packages/agents
pnpm run test:conformance                  # all three suites
bash conformance/run.sh client --scenario auth/scope-step-up   # single scenario

Also verified: pnpm vitest --run src/tests/mcp/... (188 tests) and typecheck pass with the library changes.

@changeset-bot

changeset-bot Bot commented Jun 9, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 50a900a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
agents Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@mattzcarey mattzcarey marked this pull request as ready for review June 9, 2026 21:54

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.

Open in Devin Review

@pkg-pr-new

pkg-pr-new Bot commented Jun 9, 2026

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@1722

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@1722

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@1722

create-think

npm i https://pkg.pr.new/create-think@1722

hono-agents

npm i https://pkg.pr.new/hono-agents@1722

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@1722

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@1722

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@1722

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@1722

commit: 50a900a

Matt Carey added 3 commits June 17, 2026 11:31
…use bugs

Runs the official @modelcontextprotocol/conformance suite (as used by the
MCP TypeScript SDK) against the MCP client (Agent + MCPClientManager),
McpAgent, and createMcpHandler + WorkerTransport - all hosted in workerd
via wrangler dev so the implementations are tested as they run in
production.

The client suite found two real OAuth bugs in MCPClientConnection, fixed
here:
- finishAuth ran on a freshly created transport, losing the resource
  metadata URL captured from the WWW-Authenticate header, so token
  exchange fell back to the default /token endpoint
- reconnecting after a mid-session 401 (scope step-up, token revocation)
  failed permanently with 'Already connected to a transport'

Closes #1721
…er-sse-polling

--suite runs scenarios in parallel against the worker, which makes the
timing-sensitive server-sse-polling scenario record different results on
fast vs slow machines (zero checks locally, SHOULD-level warnings on CI).
Sequential single-scenario runs are deterministic.

server-sse-polling is baselined: McpAgent's internal transport does not
implement SEP-1699 priming events or the retry field (tracked in #1723),
and the harness probes with protocol 2025-03-26 for which the pinned SDK
deliberately suppresses priming on the WorkerTransport variant.
sse-retry flaked on CI: connection retry backoff plus the SDK's 5s SSE
retry interval can exceed the worker's 25s waitForReady and the
conformance CLI's default 30s scenario timeout on cold runners. Raise
both (50s / 90s).
@mattzcarey mattzcarey force-pushed the mcp-client-conformance branch from 5e4190d to 50a900a Compare June 17, 2026 10:33
@mattzcarey mattzcarey merged commit 9f8e14b into main Jun 17, 2026
7 checks passed
@mattzcarey mattzcarey deleted the mcp-client-conformance branch June 17, 2026 10:49
@github-actions github-actions Bot mentioned this pull request Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add MCP client conformance tests using @modelcontextprotocol/conformance

2 participants