feat(registry): unify agent tools behind an AgentCore Gateway (managed MCP aggregation + dual-sided auth, any substrate)

[isadeks]

Summary

Adopt Amazon Bedrock AgentCore Gateway as a managed, single, secure entry point that gives the agent access to tools — regardless of compute substrate (AgentCore microVM and ECS/Fargate). Gateway is a fully managed AI gateway that converts APIs, Lambda functions, and existing services into Model Context Protocol (MCP)-compatible tools, aggregates multiple targets behind one unified virtual MCP server (a single consolidated tools/list), and manages both inbound and outbound authentication so we stop hand-rolling per-tool credential handling and per-thread .mcp.json wiring in the agent container.

Rather than each agent task connecting to each tool/MCP server directly (with the tool's credential living inside the container), the agent connects once to its gateway endpoint; the gateway does protocol translation, credential injection per target, and semantic tool selection.

What AgentCore Gateway gives us (per AWS docs)

• Targets → MCP tools. Three target categories: MCP targets (aggregation mode — Lambda functions, API Gateway REST APIs, OpenAPI specs, Smithy models, remote MCP servers, and built-in integration templates/connectors incl. 1-click Slack/Jira/Asana/Zendesk/Salesforce), HTTP targets (passthrough to other agents/A2A/HTTP endpoints), and inference targets (model routing). Our near-term use is MCP targets.
• Unified virtual MCP server. MCP targets combine into one tools/list; clients see a single endpoint. Supports capability synchronization and semantic tool search (find the right tool by task context → use many tools without bloating the prompt).
• Managed dual-sided auth — inbound (verify the caller) and outbound (reach the target), in one service.
• Serverless + observable — no infra to manage; built-in auditing.

Auth model (grounded in the docs — OAuth is NOT central)

• Inbound authorizer (agent → gateway) — the gateway supports four types: OAuth (JWT), IAM (AWS Signature Version 4), authenticate-only, and no authorization. So inbound can be plain AWS IAM/SigV4 — no OAuth required. (The earlier spike used a JWT authorizer + a Cognito M2M token; IAM SigV4 is the simpler substrate-portable default worth evaluating first.)
• Outbound credential provider (gateway → target) — depends on target type: Lambda/Smithy targets use the gateway's execution role (IAM) — no per-tool credential at all; OpenAPI/MCP-server targets attach an AgentCore credential provider holding an API key or OAuth, or use IAM SigV4, or no authorization for public endpoints.

So OAuth is one outbound credential option among several, relevant only when a specific target requires it — not a property of Gateway or of this feature. Most targets we'd add (Lambda tools, IAM/API-key services) never touch OAuth.

Requirements

1. Substrate-agnostic agent access (hard requirement). The agent must reach its gateway-federated tools on both AgentCore microVM and ECS/Fargate. The inbound credential must be portable across substrates — an IAM SigV4 signature (task role) or a token minted from a secret — NOT an AgentCore-runtime-only workload-identity primitive. Verify end-to-end on each substrate (--context compute_type=ecs and the default): the agent authenticates to the gateway and a tools/list + tool call succeeds. Add the required grant (InvokeGateway / SigV4) to whichever substrate's task role lacks it.
2. Registry + config model for N targets. Per-workspace (or per-deployment) registry holds the gateway id/url + per-target metadata including inbound authorizer type and each target's outbound credential type, so tools onboard declaratively without bespoke wiring. The agent's MCP-config writer points at the gateway URL; the agent authenticates to the gateway (not to each tool).
3. Onboarding branches by target auth type (no universal consent flow):
   • IAM / execution-role targets (Lambda, Smithy, IAM-signed HTTP): register the target — no credential prompt, no consent.
   • API-key targets: vault the key once — no consent, no callback.
   • OAuth targets (only if a specific target needs it): attach an OAuth credential provider; a 3-legged (authorization-code) target additionally needs a one-time admin consent + provider-side callback-URL registration + the complete-resource-token-auth finalizer (see appendix). This path applies to that target only.
4. Semantic tool search — evaluate turning it on as tool count grows, to keep the agent's prompt small.
5. Operational hardening — provisioning idempotency, credential-provider/target lifecycle + cleanup, and (for any OAuth targets) the gotchas in the appendix.

Appendix — reference spike (OAuth 3LO target; reference only, do NOT continue off it)

A prior spike branch (feat/agentcore-gateway-mcp, pushed) took Gateway end-to-end against one 3-legged-OAuth hosted MCP target and reached a working federated endpoint (GO). It is Linear-coupled and out of scope — use it as a reference for the mechanism, not a base:

• Reusable, provider-neutral: per-workspace gateway provisioning shape; agent minting a Cognito M2M JWT for a JWT-authorizer gateway (agent/src/gateway_auth.py — substrate-neutral: reads a secret + presents the token, no runtime-only calls); CDK wiring pattern for the gateway role + M2M client (context-gated/additive); the registry gateway_url → channel_metadata → MCP-config-writer plumbing.
• 3LO-target-specific facts (only relevant if we add an OAuth authorization-code target): the target sits at CREATE_PENDING_AUTH until the CLI calls complete-resource-token-auth (data-plane bedrock-agentcore) with --user-identifier + the --session-uri PAR handle from the consent URL; a stuck target can't be deleted until it times out (~10 min); PAR request_uri expires ~10 min; provider callback UUIDs survive on the OAuth app after provider deletion (Invalid redirect_uri if stale); aws-cli ≥ 2.35 needed for the 3LO control-plane fields. Fresh-install consent only — 3LO app identity can't be retrofitted onto an already-installed app.

Full spike findings: docs/design/AGENTCORE_GATEWAY_MCP_SPIKE.md on that branch.

Latency expectations

AWS publishes no fixed latency figures for Gateway; the documented principle is that "the overall latency of your gateway is largely determined by the latency of the underlying tools." So budget latency as per-tool baseline + a roughly-constant gateway overhead, and measure — don't assume a number.

Where the gateway adds overhead vs. calling a tool directly:

• One extra network hop (agent → gateway → target) + inbound-auth verification (IAM SigV4 or JWT validation) + MCP↔target protocol translation. Roughly constant per call; small relative to real tool work (a Lambda/API round-trip), but not free for very fast tools.
• initialize / session setup — first call in a session sets up the client↔gateway session and, on first tool call, the gateway↔target session.
• Semantic tool search, when enabled — an index lookup at tool-selection time (the tradeoff: bigger tool catalogs without prompt bloat).

Levers to keep it low (all doc-backed):

• Enable MCP sessions (sessionConfiguration, default 3600s, range 900–28800s). The gateway stores the target's session ID and reuses it on subsequent tool calls, avoiding re-initialization every request; for AgentCore Runtime targets it avoids a cold-start per request. This is the single biggest gateway-side lever — the agent MUST send Mcp-Session-Id after initialize (the MCP SDK/Strands client handle this automatically).
• Lambda targets: same region as the gateway, fast cold starts, provisioned concurrency for latency-sensitive tools.
• Simple tool schemas; low-latency/high-availability REST for OpenAPI targets.
• Optional response streaming for long results.

What to measure (empirical acceptance targets for this feature — fill in from the test matrix, don't guess):

1. Cold initialize + tools/list (first call, session setup + capability sync).
2. Warm tools/call with session reuse vs. without (quantify the session win).
3. Same tool through the gateway vs. called directly (the gateway overhead delta) — per target type.
4. tools/call with vs. without semantic search enabled.
   Record p50/p90 on backgroundagent-dev; treat "direct vs. gateway delta" as the number that matters for adoption, not absolute ms.

Testing — cover multiple target TYPES (not just one OAuth server)

The prior spike only exercised a single 3LO-OAuth MCP-server target. A general Gateway feature must be tested across the target + auth combinations, since onboarding, credential handling, and latency all differ by type. Minimum matrix on backgroundagent-dev:

#	Target type	Outbound auth	What it proves
1	Lambda function	execution role (IAM)	The simplest, no-credential path; MCP↔Lambda translation; provisioned-concurrency latency lever. A throwaway "echo/add" Lambda.
2	OpenAPI / API Gateway REST	API key	API-key credential provider (vault once, no consent); REST↔MCP translation.
3	OpenAPI or HTTP	IAM SigV4	IAM-signed egress with no stored secret.
4	OpenAPI or MCP server	no authorization	Public endpoint path (dev/test only); confirms "no auth" onboarding branch.
5	Remote MCP server (2LO)	OAuth client-credentials	Machine OAuth with gateway-managed refresh, no interactive consent — the common non-Linear OAuth case.
6	Remote MCP server (3LO) (optional)	OAuth authorization-code	The full consent + callback + complete-resource-token-auth path (reference spike already covered this once).

Plus cross-cutting tests, each run on both AgentCore microVM and ECS/Fargate (the substrate requirement):

• Aggregation: attach ≥2 targets of different types to one gateway → a single tools/list returns tools from all, correctly namespaced.
• Inbound auth: IAM SigV4 authorizer AND JWT authorizer both accepted; wrong/expired credential rejected.
• Session reuse: Mcp-Session-Id reused across calls; expiry (short sessionTimeoutInSeconds) transparently re-initializes the target.
• Semantic search on: relevant tool surfaces for a task query without listing the full catalog.
• Failure modes: target down / credential invalid / gateway unreachable → the agent degrades sanely (surfaces the error, doesn't hang).
• Latency capture: run the four measurements above for each target type.

Types 1–5 are all non-Linear and cover the auth models we'll actually use; type 6 is optional since the spike proved it.

Registration UX (how an operator adds a tool)

Design principle: easy by default, doable to build. Optimize for a low-friction happy path without hiding the real mechanics or over-engineering:

• One-command happy path. bgagent gateway add-target <name> --type <t> --endpoint <e> should Just Work for the common cases (Lambda / OpenAPI / MCP-server with IAM or API-key), auto-creating the gateway on first use, choosing sane defaults (listingMode=DEFAULT, sessions on, semantic search on past N tools), and printing the resulting tool names on success.
• Interactive wizard when args are omitted. Bare bgagent gateway add-target prompts: pick type → endpoint → detect/ask auth → (only if 3LO) walk the callback+consent steps. Every prompt has a default; power users pass flags and skip it.
• Presets for known providers where the platform CAN script it (endpoint + auth-type baked in, operator supplies only the secret) — so "add the Tavily search tool" is one flag, not a schema hunt. Console-only 1-click templates get a printed deep-link + register-existing, not a dead end.
• Preflight validation before any AWS write — reachable endpoint, secret exists + parses, SigV4 target actually supports SigV4, aws-cli ≥ 2.35 for 3LO — fail early with a fix hint, not mid-provision with a stuck target.
• Idempotent + safe re-run. Re-running add-target for an existing name updates in place; a failed/PENDING_AUTH target is detected and the command offers to resume/clean up rather than erroring.
• Bounded scope (doable): ship the CLI-scriptable types first (Lambda/OpenAPI/Smithy/MCP-server × IAM/API-key/2LO — all single-call). 3LO consent and console-only templates are the harder, lower-frequency paths — support them but don't let their complexity shape the everyday flow.

Goal: a bgagent-native flow to register a target so its tools appear in the agent's unified tools/list, with the fewest steps for the common cases. UX branches by target type + outbound auth (per AWS docs), and one class is unavoidably console-only.

The good news — most registrations are effectively one step. CreateGatewayTarget in listingMode=DEFAULT implicitly synchronizes: it fetches the target's tools/list/prompts/list/resources/list and indexes them for the unified catalog + semantic search automatically. So for non-3LO targets there's no separate discovery/consent step — create → target READY → tools live.

Proposed CLI shape:

bgagent gateway add-target <name> \
  --type lambda|openapi|smithy|mcp-server|api-gateway \
  --endpoint <url|arn|schema-ref> \
  --auth iam|apikey|oauth2-cc|oauth2-3lo|none \
  [--api-key-secret <arn> | --oauth-secret <arn> | --sigv4-service <svc>] \
  [--workspace <slug>]        # per-workspace gateway; else the deployment gateway
bgagent gateway list-targets [--workspace <slug>]     # names + status + auth type
bgagent gateway sync <name>                            # re-index after the server's tools change
bgagent gateway remove-target <name>

Per-type onboarding UX (increasing friction):

Target / auth	Operator provides	Steps	Consent?
Lambda / Smithy	function ARN / model	1 (uses gateway execution role)	no
OpenAPI / API Gateway, IAM SigV4	schema/stage + --sigv4-service	1 (only if target verifies SigV4 — API GW / Lambda URL / AgentCore Runtime; not ALB/EC2)	no
any, API key	endpoint + --api-key-secret	1 (create API-key credential provider from the secret)	no
MCP server / OpenAPI, OAuth 2LO	endpoint + client-creds secret	1 (gateway manages client-credentials + refresh)	no
MCP server, OAuth 3LO	endpoint + OAuth app creds	2-phase: register callback URL on the provider's app → CreateGatewayTarget returns an authorization URL → admin consents → CLI calls CompleteResourceTokenAuth (finalizer)	yes, one-time admin
1-click provider template (Slack/Jira/Asana/Zendesk/Salesforce/…)	via AWS Console only	console wizard (API/CLI NOT supported for templates)	depends on provider

Two things the UX must surface honestly:

1. 1-click integration templates are Console-only — AWS does not expose them via API/CLI ("You can only add an integration provider template as a target through the AWS Management Console and not through the API"). So bgagent cannot provision Slack/Jira/etc. templates; for those, bgagent gateway should print the console deep-link + the values to paste, then let the operator register the resulting gateway/target id back into the ABCA registry (bgagent gateway register-existing <gateway-id>). The generic types above (Lambda/OpenAPI/Smithy/MCP-server) ARE fully CLI-scriptable via CreateGatewayTarget.
2. Only 3LO OAuth needs the consent dance — every other type is a single non-interactive call. Don't make the UX pay the 3LO callback/consent cost for API-key/IAM/2LO targets (the reference spike conflated these because it only did 3LO).

Re-sync UX: when a target's tools change, bgagent gateway sync <name> → SynchronizeGatewayTargets (202, async; poll GetGatewayTarget). Needed for DEFAULT-mode targets so semantic search + tools/list reflect the latest tools.

Status/observability in the CLI: list-targets should show each target's status (READY / CREATE_PENDING_AUTH / FAILED) and auth type, so an operator can see a stuck 3LO consent or a failed sync without dropping to the console. (A target in *_PENDING_AUTH can't be updated/deleted/synced until it resolves — surface that state clearly rather than letting a command error.)

Provisioning-role IAM (from the AWS docs) the CLI/stack needs: bedrock-agentcore:CreateGateway/GetGateway/CreateGatewayTarget/GetGatewayTarget/UpdateGatewayTarget/SynchronizeGatewayTargets + the identity actions (GetWorkloadAccessToken*, GetResourceOauth2Token, GetResourceApiKey, CompleteResourceTokenAuth) + secretsmanager:GetSecretValue + KMS for the credential vault.

[isadeks]

Reference implementation (existing spike branch) — start fresh, do NOT continue off it

A design spike branch, feat/agentcore-gateway-mcp (pushed to origin + upstream), already implements the full Gateway path end-to-end. It has real, working, unit-tested code and is the best reference for the mechanism — but it was built to federate Linear specifically, which is out of scope here (Linear is deterministic on the agent by design, ADR-016 — no agent-side MCP). So:

Treat that branch as a reference, not a base. A new implementation for this issue should be started fresh and provider-agnostic; do not continue off feat/agentcore-gateway-mcp (it would carry the Linear-MCP coupling this issue explicitly excludes).

What's directly reusable (the provider-agnostic mechanism):

• Per-workspace Gateway provisioning shape (cli/src/linear-gateway.ts): create Gateway (CUSTOM_JWT inbound) → OAuth2 credential provider → MCP target → wait-for-ready.
• Inbound auth: agent mints a Cognito M2M (client_credentials) JWT and presents it to the CUSTOM_JWT gateway (agent/src/gateway_auth.py) — AWS_IAM inbound is rejected for 3LO targets.
• CDK wiring pattern: gateway service role, Cognito M2M app-client/resource-server/secret, runtime InvokeGateway grant — all context-gated/additive.
• The complete-resource-token-auth finalizer requirement (item 1 above) and the two-phase callback→consent flow.
• The registry-row gateway_url → channel_metadata → MCP-config-writer plumbing pattern.

What is Linear-specific and must be dropped / generalized (do NOT copy verbatim):

• Hardcoded LINEAR_MCP_ENDPOINT = https://mcp.linear.app/mcp, LINEAR_AUTHORIZE/TOKEN_ENDPOINT, GATEWAY_LINEAR_SCOPES, actor=app customParameters, GATEWAY_NAME_PREFIX = 'bgagent-linear', the bgagent-linear-oauth-<slug> secret coupling — all of these become per-provider config in a general registry model (item 4 above).
• The linear-webhook-processor / linear-oauth-resolver gateway_url plumbing is Linear-path-specific; the general version keys off the registry's per-server target metadata, not the Linear resolver.

Full spike findings + gotchas live in docs/design/AGENTCORE_GATEWAY_MCP_SPIKE.md on that branch.