Repository navigation
feat(ask-demo): an example toolset whose tool asks the user a question - #48
Merged
Merged
Conversation
Contributor
Author
Model invoked interrupt
Tool invoked interrupts
|
ciaransweet
added a commit
to developmentseed/mcp-toolsets-runtime
that referenced
this pull request
Oct 20, 2026
Part 2 of #166: a toolset tool can stop mid-call and ask the user a question. One call, `ask`, with a pydantic model of the answer: ```python from pydantic import BaseModel, Field from mcp_runtime.questions import ask class Year(BaseModel): year: int = Field(ge=1940, le=2026, description="The year to download") @tool async def pick_year() -> ToolResult: """Pick a year, asking the user which.""" chosen = ask("Which year?", Year) # an instance, or None when the user declined return ToolResult(message=f"picked {chosen.year}" if chosen else "no year chosen") ``` The first time through, `ask` ends the call with the question. The agent puts it to the user, calls the tool again from the start, and this time `ask` returns the answer. It's the pattern the agent's own tools use with LangGraph's `interrupt()`. ## `ask` - **The model is the question.** Its JSON schema goes to the client, so `Field` bounds, lengths, patterns and descriptions, `Literal` choices, optionals, defaults, `date` and `HttpUrl` all reach it as the schema they imply. The reply is validated back into an instance, so a tool reads `chosen.year`. A JSON schema dict in place of the model is answered as a dict. - **How it works.** `ask` raises `Asked`, which the bridge catches and returns as the SDK's `InputRequiredResult` (`convert_result` passes it through). On the retry the bridge takes the SDK context (`context_kwarg`) and publishes its `input_responses` through a contextvar, the way `credentials.py` publishes headers. A tool never sees the exception; a test does, with `pytest.raises(Asked)`, and reaches the second half with `answered(content)`, as `header_context` gives a credential. - **One `ask` per call, form requests only.** Several things go in one model, since the SDK hands back the answers for a single round. A `url` request is something the agent can relay, but no toolset needs one yet. ## Getting the answer back #166 assumed a tool that elicits "would already raise an interrupt through our stack". It didn't: the turn crashed with `MCPError: Elicitation not supported`, because the agent's clients were no longer armed for elicitation once `list_tools` stopped going through `MCPAdapter`, and each call built its tool on a client that hadn't connected, so `langchain.mcp` never saw a protocol version that allows a pause. `with_credential_support` now arms each client (`MCPAdapter(client).client`), and `_on_its_own_session` connects before it builds. The run then pauses on `langchain.mcp`'s own value and waits for a resume in its shape. `question` and `answer` in `mcp_agent.interrupts` translate to and from the shape the `interrupt` tool already uses, so every surface handles both the same way: `{reason, message, responseSchema}` with the tool's name under `metadata.tool`, applied by `pending()` so `TurnResult`, the AG-UI outcome and `GET /threads/{id}` all carry it, and the API's existing 422 check covers it. A skip (`cancelled`) **declines**, since MCP's `cancel` means dismissed without a choice and skipping is a choice; so does a request the payload leaves out. Several requests in one round become one question with each schema under its key; a `url` request puts its address in the message. `INPUT_REQUIRED` moved from `interrupt_gate` to `interrupts`, where `question` needs it. ## Limits, on purpose - **Only the 2026-07-28 protocol is supported.** Arming tells every server the agent can answer questions. A server on that revision asks by returning the question as a result, which works. A server on an older revision asks by calling the client back mid-call, which the agent cannot answer, so that tool call fails. A server that would have skipped the question for a client that can't answer will now ask instead. `docs/CONSUMING.md` §5g says so. - **The question has no `toolCallId`.** What pauses the run names the tool, not the call. - **The tool runs again before it gets the answer.** On resume the call starts from the beginning: the tool is called without the answer, asks again, and is then called with it. That's how `langchain.mcp` works. §2 tells authors to ask before any work that should happen once. - **The web client draws a form request; the terminal doesn't.** One native input per property of `responseSchema`: text, number, email, URL or date box, checkbox, select for an enum, multiple select for an array of one. `default` and `description` are used, and `required`, `minimum`, `maximum`, `minLength`, `maxLength`, `pattern` and `format` are checked by the browser before the answer is sent (the API checks it again). An optional field comes as pydantic's `anyOf: [type, null]`, which the form unwraps. The form sits under the tool call that asked, paired by tool name with the call of that tool that has no result yet, so an error afterwards lands below it rather than pushing it down. A round that asked several things at once nests an object per request, which the form doesn't draw, so it offers only **skip**; before this a tool's question left the thread unusable, with nothing shown and sending blocked. The terminal cancels a question that isn't a choice, without prompting. ## Checked - `tests/mcp_agent/test_elicitation.py` runs a real MCP server over streamable HTTP whose tool is a LangChain tool through `to_mcp_tool`, so the round trip crosses the bridge, `with_credential_support`, `list_tools` and the state-wired agent. It checks that the run stops on the tool's question (with its tool name), that an answer reaches the tool, and that a skip reaches it as a decline. Unit tests cover: - the bridge returning the question, and the answers reaching the tool on the retry, accepted and declined; - several requests in one round; - `url` requests, alone and together; - a request the answer leaves out being declined; - the AG-UI interrupt that goes out; - the API refusing an answer that doesn't fit with 422. - Live, against Mistral, through the bundled app, over the four tools of developmentseed/mcp-toolsets#48: each question pauses the run with `metadata.tool` and draws as a form under its call; answers round-trip to the tool (`Downloaded era5 for 2024. Told a@b.c.`, `Subscribed to era5: monthly from 2026-11-02, as every change.`); a skip reaches the tool as `decline`; the browser refuses a bad email or label before sending. - `./scripts/lint`, `./scripts/test` (603 passed), `./scripts/build-js`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ciaransweet
force-pushed
the
feat/ask-demo
branch
2 times, most recently
from
November 10, 2026 14:01
e005179 to
5c4d40e
Compare
ciaransweet
marked this pull request as ready for review
November 10, 2026 14:02
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tion Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ciaransweet
force-pushed
the
feat/ask-demo
branch
from
November 10, 2026 16:01
5c4d40e to
438da9f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An example toolset,
ask-demo, whose tools stop mid-call to ask the user a question. It shows the pattern developmentseed/mcp-toolsets-runtime#201 adds for toolset authors:askends the call with the question (the model's JSON schema) the first time; the agent puts it to the user and calls the tool again, from the start, andaskreturns the reply as an instance of the model. Four tools between them use every kind of field a question can have: a bounded integer and an optional email (download), aLiteralselect with a default, adateand a checkbox (subscribe), a multiple select (export), and anHttpUrlwith a pattern-checked label (register_webhook).EmailStrneedspydantic[email], which the toolset declares. The tests reach the second half withanswered, the waycredential-demo's give a credential withheader_context.Scaffolded with
uv run mcp-toolset new ask-demo; the README lists it besidehelloandcredential-demo.Draft until the runtime releases
mcp_runtime.questionsis on the runtime's #201, not yet released. Against the pinned 0.11.0 the contract test fails onask-demowithModuleNotFoundError. Against the branch,tests/test_contract.pyand the toolset's tests pass (27 passed). Once #201 is merged and released, bump the pin here and this is ready.🤖 Generated with Claude Code