From 5bd80b85c0cc42ea0a10ebcd82ca1c26806a8732 Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Wed, 14 Oct 2026 14:29:35 +0200 Subject: [PATCH 01/10] Python: Add Agent Framework consumer skill --- .github/CODEOWNERS | 3 + .github/copilot-instructions.md | 13 ++ README.md | 18 ++ python/AGENTS.md | 9 + skills/README.md | 35 ++++ .../agent-framework-builder-python/SKILL.md | 88 +++++++++ .../references/agent-builder.md | 174 ++++++++++++++++++ .../references/declarative.md | 89 +++++++++ .../references/evaluation.md | 92 +++++++++ .../references/harness-agent-builder.md | 171 +++++++++++++++++ .../references/hosting-and-deployment.md | 150 +++++++++++++++ .../references/integrations.md | 129 +++++++++++++ .../references/observability.md | 43 +++++ .../references/workflow-builder.md | 143 ++++++++++++++ 14 files changed, 1157 insertions(+) create mode 100644 skills/README.md create mode 100644 skills/agent-framework-builder-python/SKILL.md create mode 100644 skills/agent-framework-builder-python/references/agent-builder.md create mode 100644 skills/agent-framework-builder-python/references/declarative.md create mode 100644 skills/agent-framework-builder-python/references/evaluation.md create mode 100644 skills/agent-framework-builder-python/references/harness-agent-builder.md create mode 100644 skills/agent-framework-builder-python/references/hosting-and-deployment.md create mode 100644 skills/agent-framework-builder-python/references/integrations.md create mode 100644 skills/agent-framework-builder-python/references/observability.md create mode 100644 skills/agent-framework-builder-python/references/workflow-builder.md diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 445c245f17e..78bcaa7842d 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -12,3 +12,6 @@ # .NET owners /dotnet/ @eavanvalkenburg @SergeyMenshykh @rogerbarreto @westey-m @moonbox3 @baywet @peibekwe + +# Public Python consumer skill +/skills/agent-framework-builder-python/ @eavanvalkenburg @SergeyMenshykh @rogerbarreto @westey-m @moonbox3 @jpalvarezl @sophia-ramsey @TaoChenOSU @giles17 diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 96d92f163fa..5de768027a7 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -7,6 +7,19 @@ Microsoft Agent Framework - a multi-language framework for building, orchestrati - `python/` - Python implementation → see [python/AGENTS.md](../python/AGENTS.md) - `dotnet/` - C#/.NET implementation → see [dotnet/AGENTS.md](../dotnet/AGENTS.md) - `docs/` - Design documents and architectural decision records +- `skills/` - Public, installable Agent Skills for developers using Agent Framework + +## Public Consumer Skills + +The skills under `skills/` are distribution artifacts for developers building Agent Framework applications. They are +not project skills for contributors working in this repository: + +- Do not load or invoke them as instructions for normal Agent Framework repository work. +- Keep them outside `.github/skills`, `.agents/skills`, and `.claude/skills` so coding agents do not auto-discover them. +- When a change affects public APIs, packages, samples, configuration, evaluation, integrations, or hosting guidance, + review and update the corresponding consumer skill as maintained product documentation. +- Read a consumer skill during repository work only when that maintenance check is relevant; do not let its standalone + application defaults override repository-specific instructions. ## Architectural Decision Records (ADRs) diff --git a/README.md b/README.md index c5b3b07d9b5..5d20026ec9d 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,7 @@ Explore new MAF capabilities and real implementation patterns on the [official b - [Quickstart](#quickstart) - [Basic Agent - Python](#basic-agent---python) - [Basic Agent - .NET](#basic-agent---net) +- [Agent Skills for Coding Agents](#agent-skills-for-coding-agents) - [More Examples & Samples](#more-examples--samples) - [Community & Feedback](#community--feedback) - [Troubleshooting](#troubleshooting) @@ -161,6 +162,23 @@ AIAgent agent = Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Framework.")); ``` +## Agent Skills for Coding Agents + +Install the Python Agent Framework builder skill to give a compatible coding agent current guidance for choosing and +building agents, harnesses, workflows, evaluations, integrations, and hosting: + +```bash +gh skill install microsoft/agent-framework agent-framework-builder-python +``` + +The cross-agent `skills` CLI is also supported: + +```bash +npx skills add microsoft/agent-framework --skill agent-framework-builder-python +``` + +See the [consumer skills catalog](./skills/) for details. + ## More Examples & Samples ### Python diff --git a/python/AGENTS.md b/python/AGENTS.md index 86f19e45714..c05ca07b000 100644 --- a/python/AGENTS.md +++ b/python/AGENTS.md @@ -18,11 +18,17 @@ Instructions for AI coding agents working in the Python codebase. - `pull-requests` — writing PR descriptions (template) and handling/resolving PR review comments - `agent-framework-py-release` — Python release PR workflow, CHANGELOG-driven package bumps, lifecycle version rules, and dependency-floor validation +**Public consumer skill** (`../skills/agent-framework-builder-python/`) — installable guidance for developers building +Python Agent Framework applications. It is a distribution artifact, not a repository skill: do not invoke it for +normal framework development. Review it as documentation when changes affect the public developer experience. + ## Maintaining Documentation When making changes to a package, check if the following need updates: - The package's `AGENTS.md` file (adding/removing/renaming public APIs, architecture changes, import path changes) - The agent skills in `.github/skills/` if conventions, commands, or workflows change +- The public Python consumer skill in `../skills/agent-framework-builder-python/` if public APIs, package names or + maturity, canonical samples, configuration, evaluation, integrations, or hosting guidance change At the end of every run, re-read `AGENTS.md` and the relevant skill files and update any guidance that the conversation revealed to be out of date, @@ -31,6 +37,9 @@ the user confirmed, etc.). **Before adding a new principle or rule, ask the user whether they want it captured as a durable principle** — do not invent team norms from a single conversation without explicit confirmation. +Only inspect the public consumer skill when the change falls into its maintenance scope. Do not load it as an active +skill or apply its standalone-project defaults to this monorepo. + ## Terminology - **Avoid "GA" for Agent Framework code.** Reserve *GA* for hosted services diff --git a/skills/README.md b/skills/README.md new file mode 100644 index 00000000000..da7d2406c45 --- /dev/null +++ b/skills/README.md @@ -0,0 +1,35 @@ +# Microsoft Agent Framework consumer skills + +This directory contains installable Agent Skills for developers building applications with Microsoft Agent Framework. +These are public distribution artifacts, not contributor instructions for working on the Agent Framework repository. + +## Available skills + +| Skill | Purpose | +| --- | --- | +| `agent-framework-builder-python` | Build, modify, evaluate, integrate, and host Python Agent Framework applications. | + +## Install + +Preview and install with GitHub CLI: + +```bash +gh skill preview microsoft/agent-framework agent-framework-builder-python +gh skill install microsoft/agent-framework agent-framework-builder-python +``` + +Or use the cross-agent skills CLI: + +```bash +npx skills add microsoft/agent-framework --skill agent-framework-builder-python +``` + +## Repository maintenance contract + +- Coding agents working on Agent Framework itself must not treat the skills in this directory as repository + instructions or automatically load them for normal framework development. +- Keep the skills outside `.github/skills`, `.agents/skills`, and `.claude/skills` so repository agents do not + discover them as project skills. +- Review the Python consumer skill whenever a change affects public APIs, package names or maturity, canonical samples, + configuration, evaluation, integrations, or hosting guidance. +- Update only the affected guidance. Do not copy internal contributor conventions into the consumer skill. diff --git a/skills/agent-framework-builder-python/SKILL.md b/skills/agent-framework-builder-python/SKILL.md new file mode 100644 index 00000000000..02a6183e322 --- /dev/null +++ b/skills/agent-framework-builder-python/SKILL.md @@ -0,0 +1,88 @@ +--- +name: agent-framework-builder-python +description: Use this skill when building, modifying, reviewing, evaluating, integrating, or hosting Python applications with Microsoft Agent Framework. Use it for Agent Framework agents, harness agents, workflows, orchestrations, declarative definitions, tools, sessions, middleware, evaluation, observability, integrations, or hosting, even when the user has not yet chosen the correct framework pattern. +license: MIT +compatibility: Agent Skills-compatible coding agents; examples use Python 3.13 and current Microsoft Agent Framework packages. +metadata: + version: "0.3" + project: agent-framework + language: python +--- + +# Agent Framework builder for Python + +Use this skill as the entry point for Python Agent Framework application work. Choose the smallest framework pattern +that preserves the required behavior, load only the relevant references, and validate against current APIs instead of +relying on remembered preview-era syntax. + +## Workflow + +1. Inspect the existing project, Python version, dependency manager, and installed Agent Framework packages. +2. Choose the simplest matching path from the table below. +3. Resolve design questions only when they materially change the implementation. +4. Read only the linked reference for the selected path. +5. Implement the smallest coherent change and run the narrowest validation that exercises it. + +## Core abstractions + +- A **chat client** implements the model-facing `SupportsChatGetResponse` protocol. It sends messages and options to a + model provider and returns chat responses or streaming updates. +- An **agent** implements the `SupportsAgentRun` protocol. The standard `Agent` implementation combines a chat client + with instructions, tools, middleware, context providers, and session handling behind one `run(...)` interface. +- A **harness agent** is a regular `Agent` assembled by `create_harness_agent`. It adds an opinionated set of context + providers and middleware; it does not introduce a separate agent abstraction. +- A **workflow** coordinates explicit execution paths. Graph workflows and orchestration builders can use multiple + `SupportsAgentRun` implementations as participants or executors and combine them sequentially, concurrently, through + handoffs, group chat, Magentic coordination, or custom routing. + +## User interface options + +- **DevUI** is a local development, testing, and debugging tool for interacting with agents and workflows. Never run + or deploy DevUI as a production application host. +- **AG-UI** is the application-facing protocol integration for connecting an agent or workflow to an AG-UI-compatible + frontend. Use the `agent-framework-ag-ui` package when the product needs streaming UI events, tool interactions, + shared state, interrupts, or resume behavior. + +## Choose the build path + +| Path | Choose it when | Details | +| --- | --- | --- | +| Plain agent | One model-driven actor owns the task and tools, sessions, context, and middleware are sufficient. | [Plain agents](references/agent-builder.md) | +| Harness agent | The product needs an operating loop with todos, modes, memory, approvals, skills, optional file access, background agents, shell, compaction, or repeated execution. | [Harness agents](references/harness-agent-builder.md) | +| Workflow or orchestration | Explicit stages, routing, parallelism, multiple participants, checkpointing, human input, or intermediate outputs are central to correctness. | [Workflows](references/workflow-builder.md) | +| Declarative definition | YAML or JSON should be the source of truth for a supported agent or workflow. | [Declarative definitions](references/declarative.md) | +| Evaluation | Expected outputs, tool calls, failed runs, or quality regressions are the main concern. | [Evaluation](references/evaluation.md) | +| Observability | The task is diagnosing or monitoring runtime behavior with traces, logs, and metrics. | [Observability](references/observability.md) | +| Integration | The task is choosing or wiring a provider, protocol, memory store, tool service, UI, or platform extension. | [Integrations](references/integrations.md) | +| Hosting and deployment | Working Agent Framework code needs a process host, protocol endpoint, identity contract, or deployment shape. | [Hosting and deployment](references/hosting-and-deployment.md) | + +## Defaults + +- Agent Framework supports Python 3.11 and later. Build new environments with the latest supported Python release, or + at least the previous release (N-1), unless a required dependency has a narrower compatibility range. +- Install the narrow Agent Framework distribution needed by the application instead of defaulting to the + `agent-framework` meta-package. +- Agent Framework does not automatically read `.env` files. If the project uses one, call `load_dotenv()` explicitly + or use the selected API's documented `env_file_path` or equivalent initialization option. + +## Current documentation lookup + +Use local guidance for stable decisions and common patterns. For provider matrices, package maturity, hosted tools, +deployment details, or unfamiliar APIs, consult current sources before generating code: + +1. Search Microsoft Learn with `microsoft_docs_search`. +2. Fetch the relevant page with `microsoft_docs_fetch`. +3. Use `microsoft_code_sample_search` when a current implementation example is needed. +4. If Learn MCP is unavailable, use the equivalent `mslearn` command: + +| MCP tool | CLI fallback | +| --- | --- | +| `microsoft_docs_search(query="...")` | `npx @microsoft/learn-cli search "..."` | +| `microsoft_docs_fetch(url="...")` | `npx @microsoft/learn-cli fetch "..."` | +| `microsoft_code_sample_search(query="...", language="python")` | `npx @microsoft/learn-cli code-search "..." --language python` | + +Canonical starting points: + +- [Agent Framework overview](https://learn.microsoft.com/agent-framework/overview/) +- [Your first agent](https://learn.microsoft.com/agent-framework/get-started/your-first-agent) +- [Python samples](https://github.com/microsoft/agent-framework/tree/main/python/samples) diff --git a/skills/agent-framework-builder-python/references/agent-builder.md b/skills/agent-framework-builder-python/references/agent-builder.md new file mode 100644 index 00000000000..1036862fd9a --- /dev/null +++ b/skills/agent-framework-builder-python/references/agent-builder.md @@ -0,0 +1,174 @@ +# Plain agent builder + +Use this path for Python agents whose behavior can be expressed with one `Agent`, a model or remote-agent client, tools, +sessions, context providers, and middleware. + +## Requirement intake + +Capture: + +1. Goal and user-visible behavior. +2. Input and output shape, including streaming or structured output. +3. Model provider, authentication, and whether the definition is application-owned or service-managed. +4. Tools, hosted tools, and side effects that require approval. +5. Session, history, retrieval, memory, and compaction requirements. +6. Middleware, runtime context, observability, and evaluation criteria. +7. Local, self-hosted, protocol-hosted, durable, or Foundry-hosted deployment. + +Ask only about unknowns that materially affect the design. + +## Minimal project + +```text +my-agent/ + pyproject.toml + agent.py +``` + +For the canonical Foundry-backed starting point, use the package manager selected by the project or user: + +```bash +# uv +uv init --bare --python 3.13 +uv add agent-framework-foundry azure-identity + +# pip +python -m pip install agent-framework-foundry azure-identity +``` + +Install only the distributions required by the selected provider and capabilities. Add prerelease resolution only when +one of those distributions is prerelease. + +## Minimal Foundry agent + +Use environment variables in real applications: + +```python +from __future__ import annotations + +import asyncio +import os + +from agent_framework import Agent +from agent_framework.foundry import FoundryChatClient +from azure.identity import AzureCliCredential + + +async def main() -> None: + agent = Agent( + client=FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + credential=AzureCliCredential(), + ), + name="my-agent", + instructions="Describe the agent behavior here.", + ) + print(await agent.run("Hello")) + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +Agent Framework does not automatically load `.env` files. Add `python-dotenv` and call `load_dotenv()` only when the +application intentionally uses a `.env` file. + +## Provider selection + +| Need | Distribution and API | +| --- | --- | +| Foundry project model with application-owned instructions and tools | `agent-framework-foundry`, `FoundryChatClient` | +| Interact with an existing agent of any supported type already defined in Foundry | `agent-framework-foundry`, `FoundryAgent` | +| OpenAI and OpenAI-compatible endpoints with Responses semantics | `agent-framework-openai`, `OpenAIChatClient` | +| OpenAI and OpenAI-compatible endpoints with Chat Completions semantics | `agent-framework-openai`, `OpenAIChatCompletionClient` | +| Other model or agent service | Its current provider distribution and documented client or agent | + +`FoundryAgent` connects to and interacts with an agent that already exists in Foundry. It does not create, deploy, +version, update, or otherwise manage that agent definition; use the appropriate Foundry management APIs or deployment +tools for those operations. + +Use current provider documentation before assuming hosted-tool, storage, streaming, or structured-output parity. + +## Tools and approval + +- Prefer focused, typed, non-interactive tool functions. +- Use `@tool` or `FunctionTool` when explicit schema or approval behavior is needed. +- Tool approval defaults to `never_require`; set `approval_mode="always_require"` for side effects. +- Treat writes, sends, purchases, destructive operations, shell commands, and privileged external calls as side effects. +- Keep return values structured when downstream code or evaluation consumes them. +- Use hosted tools only with providers that document support. + +For parameter descriptions, use normal annotations and current schema guidance rather than inventing provider-specific +JSON schemas. + +## Sessions + +Use an `AgentSession` for multi-turn state: + +```python +session = agent.create_session() +await agent.run("Remember Amsterdam.", session=session) +response = await agent.run("What city should you remember?", session=session) +``` + +- Do not reuse a session concurrently unless the provider and application explicitly support it. +- Treat persisted session state as sensitive data. + +## History + +Choose one primary history owner: + +- **Application-managed history** uses a `HistoryProvider` to load prior messages and persist new messages in + `AgentSession` state or an external store. +- **Service-managed history** is available on clients that support a `store` option. With `store=True`, the provider + stores the conversation and returns a service continuation identifier that Agent Framework keeps in + `AgentSession.service_session_id`; later turns can send new input against that stored conversation instead of + resending the full transcript. +- With `store=False`, the provider does not retain the conversation. Use a `HistoryProvider` or supply the required + prior messages when multi-turn context is needed. +- When the chat client does not support service-managed storage and no configured `HistoryProvider` reads history, + `Agent` adds an `InMemoryHistoryProvider` by default so an `AgentSession` retains conversation history for the + current process. Replace it with a persistent history provider when history must survive process restarts or scale + across instances. + +- Not every provider supports `store`, and service-managed continuation and branching behavior varies by provider. +- Avoid combining service-managed storage with a load-enabled history provider unless the provider documentation + explicitly describes that combination; otherwise the model may receive duplicate history. +- Treat stored messages and service continuation identifiers as sensitive data. +- Add compaction when conversations or tool loops can exceed the model context window. + +## Context providers + +Use a `ContextProvider` when instructions, messages, tools, retrieval results, memory, or other context must be supplied +dynamically for a run or session. + +- Keep conversation history ownership in a `HistoryProvider`; context providers serve a different extension point. +- Prefer scoped context providers over concatenating changing data into static agent instructions. +- Define when the provider loads and persists data, and keep its session-state keys distinct from other providers. +- Treat retrieved and persisted context as sensitive and enforce tenant and user boundaries outside the model. + +## Middleware + +Choose the narrowest middleware layer: + +- agent middleware for run-level policy, termination, result overrides, and shared state; +- chat middleware for model request and response behavior; +- function middleware for tool invocation behavior. + +In Python middleware, call `await call_next()` without passing the context again. + +## Validation + +1. Import and construct the client and agent without invoking the model where practical. +2. Validate required environment variables with clear errors. +3. Unit-test deterministic tools, schemas, context providers, and middleware. +4. Add an opt-in live smoke test only when provider access is available. +5. Exercise a second turn when session behavior is required. +6. Check approval behavior with at least one side-effecting tool. + +Current examples: + +- [Minimal agent](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/01_hello_agent.py) +- [Function tool](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/02_add_tools.py) +- [Multi-turn session](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/03_multi_turn.py) diff --git a/skills/agent-framework-builder-python/references/declarative.md b/skills/agent-framework-builder-python/references/declarative.md new file mode 100644 index 00000000000..240ce20d752 --- /dev/null +++ b/skills/agent-framework-builder-python/references/declarative.md @@ -0,0 +1,89 @@ +# Declarative agents and workflows + +Use declarative construction when YAML or JSON should be the source of truth for a supported agent or workflow +definition. Keep credentials, custom tools, transports, authorization, and application services in code. + +## Package and maturity + +`agent-framework-declarative` is a released distribution: + +```bash +uv add agent-framework-declarative +``` + +Add the selected provider distribution separately. The package being released does not make every API stable: + +- Declarative agent loading through `AgentFactory` is experimental. +- Declarative workflow construction through `WorkflowFactory` is not feature-stage decorated. + +Check current package and feature maturity before making compatibility commitments. + +## Declarative agents + +Use `AgentFactory` to create an agent from inline YAML or a file: + +```python +import os + +from agent_framework.declarative import AgentFactory +from agent_framework.foundry import FoundryChatClient +from azure.identity import AzureCliCredential + + +client = FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + credential=AzureCliCredential(), +) +factory = AgentFactory(client=client) +agent = factory.create_agent_from_yaml_path("agent.yaml") +``` + +The application remains responsible for provider configuration, credentials, allowed bindings, connections, and +tools. `AgentFactory` emits the experimental feature warning. + +See [declarative agents](https://learn.microsoft.com/agent-framework/agents/declarative). + +## Declarative workflows + +Use `WorkflowFactory` for supported YAML workflow actions: + +```python +from agent_framework.declarative import WorkflowFactory + + +factory = WorkflowFactory() +workflow = factory.create_workflow_from_yaml_path("workflow.yaml") +``` + +Supply application-owned agents and bindings explicitly. Provide an HTTP or MCP handler when the workflow uses those +actions; build should fail rather than silently dispatch through an unconfigured transport. + +See [declarative workflows](https://learn.microsoft.com/agent-framework/workflows/declarative). + +## When to stay code-first + +Prefer `Agent`, a functional workflow, `WorkflowBuilder`, or an orchestration builder when: + +- custom Python logic is the primary behavior; +- direct typing and refactoring matter more than configuration portability; +- the required executor, action, provider, tool, or middleware is not represented by the declarative schema; +- dynamic routing would make the YAML harder to validate than code. + +Do not invent unsupported YAML kinds. Use current schema and examples. + +## Security and validation + +- Treat declarative definitions as executable application configuration. +- Load definitions only from trusted or authenticated sources. +- Keep secrets and credentials outside YAML. +- Keep environment exposure restricted to explicitly supplied configuration by default. +- Allowlist HTTP and MCP destinations. +- Register only the tools, agents, and services the definition may invoke. +- Use custom handlers for authentication, retries, SSRF controls, and connection ownership. +- `DefaultHttpRequestHandler` limits wire and decoded response bodies to 10 MiB by default, including intermediate + redirect and authentication responses. Set `max_response_body_bytes` when the host requires a different positive + bound, and do not use response hooks or transports that buffer the body before the handler enforces it. +- Validate at startup and fail clearly on unknown providers, actions, tools, variables, or resources. +- Test declarative agents and workflows with the same approval, evaluation, and telemetry policies as code-first + construction. diff --git a/skills/agent-framework-builder-python/references/evaluation.md b/skills/agent-framework-builder-python/references/evaluation.md new file mode 100644 index 00000000000..f528dea7d11 --- /dev/null +++ b/skills/agent-framework-builder-python/references/evaluation.md @@ -0,0 +1,92 @@ +# Evaluation + +Use evaluation to turn expected behavior, failures, and user feedback into repeatable checks. + +The local evaluation APIs and Foundry evaluation APIs are currently experimental. Keep their use isolated and review +feature maturity before exposing them as a long-term application contract. + +## Capture a useful regression case + +Record: + +- input prompt or triggering event; +- current and expected output; +- expected tool calls and important arguments; +- relevant conversation, session, or workflow state; +- provider and model configuration; +- agent or workflow version; +- user feedback and the behavior it implies. + +Translate feedback into an expected output, expected tool call, context fixture, or custom evaluator instead of +inventing an unsupported framework field. + +## Diagnose before changing + +Classify the failure: + +1. deterministic code, schema, or tool bug; +2. wrong tool selection or approval behavior; +3. missing context, retrieval, memory, or history; +4. ambiguous instructions or output contract; +5. workflow routing, state, checkpoint, or output selection; +6. provider or client configuration; +7. hosting or runtime environment; +8. model limitation after application-layer causes are excluded. + +Change the lowest layer that caused the failure. + +## Local evaluation + +```python +from agent_framework import LocalEvaluator, evaluate_agent, keyword_check, tool_called_check + + +evaluator = LocalEvaluator( + keyword_check("weather"), + tool_called_check("get_weather"), +) + +results = await evaluate_agent( + agent=agent, + queries=["What is the weather in Amsterdam?"], + expected_output=["A concise weather answer."], + evaluators=evaluator, +) +``` + +Useful APIs include: + +- `EvalItem`, `EvalResults`, `Evaluator`, and `LocalEvaluator`; +- `@evaluator` for custom checks; +- `ExpectedToolCall`; +- `evaluate_agent(...)` and `evaluate_workflow(...)`; +- `keyword_check`, `tool_called_check`, `tool_calls_present`, and tool-argument checks. + +Use exact string comparison only when exact wording is part of the contract. Prefer deterministic checks for tools, +schemas, required facts, and output structure before adding model-based graders. + +See [agent evaluation](https://learn.microsoft.com/agent-framework/agents/evaluation). + +## Foundry evaluation + +Use `FoundryEvals`, `evaluate_foundry_target`, or `evaluate_traces` when cloud evaluators, dashboards, or trace-derived +evaluation are required. These APIs are experimental. + +Keep local deterministic checks for fast inner-loop and CI coverage. Treat prompt optimization and continuous +monitoring as platform workflows rather than implicit Agent Framework behavior. + +## Improvement loop + +1. Reproduce the failure. +2. Add or update a regression case. +3. Fix the responsible layer. +4. Run deterministic checks. +5. Run model-based or Foundry evaluators only where they add signal. +6. Compare results. +7. Keep the case after the fix. + +Current examples: + +- [Local agent evaluation](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/evaluation/evaluate_agent.py) +- [Expected outputs and tool calls](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/evaluation/evaluate_with_expected.py) +- [Workflow evaluation](https://github.com/microsoft/agent-framework/blob/main/python/samples/03-workflows/evaluation/evaluate_workflow.py) diff --git a/skills/agent-framework-builder-python/references/harness-agent-builder.md b/skills/agent-framework-builder-python/references/harness-agent-builder.md new file mode 100644 index 00000000000..5b405c49504 --- /dev/null +++ b/skills/agent-framework-builder-python/references/harness-agent-builder.md @@ -0,0 +1,171 @@ +# Harness agent builder + +Use this path when the application needs a batteries-included agent operating loop rather than an explicit workflow +graph. + +`create_harness_agent` is released. Some capabilities it can enable are still experimental or prerelease, so assess +them independently instead of labeling the entire harness experimental. + +The factory is a convenience composition over the regular `Agent` building blocks, not a separate runtime. Both +directions are valid: + +- Start with `create_harness_agent` to get the full opinionated feature set, then disable or replace capabilities the + application does not need. +- Start with `Agent` and add the same instructions, chat middleware, agent middleware, context providers, history, + compaction, approval, and loop behavior explicitly. + +Both approaches produce a normal agent implementing `SupportsAgentRun`; choose based on whether fast assembly or +fine-grained composition is more important. + +## When to choose a harness + +Choose `create_harness_agent` when the agent should: + +- plan work with todos and plan/execute modes; +- keep file-backed, session-scoped memory; +- apply tool approval and standing approval rules; +- optionally load Agent Skills or use provider-supported web search; +- optionally access files, delegate to background agents, execute shell commands, or loop until work completes; +- expose the normal `Agent` interface to callers. + +Use a workflow when graph topology, routing, checkpointing, or caller-visible intermediate outputs define correctness. + +## Minimal setup + +```python +from __future__ import annotations + +import asyncio +import os + +from agent_framework import create_harness_agent +from agent_framework.foundry import FoundryChatClient +from azure.identity import AzureCliCredential + + +async def main() -> None: + agent = create_harness_agent( + client=FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + credential=AzureCliCredential(), + ), + name="my-harness-agent", + agent_instructions="Help users plan and complete multi-step tasks.", + disable_file_memory=True, + disable_web_search=True, + ) + session = agent.create_session() + print(await agent.run("Plan a weekend trip.", session=session)) + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +The session is important when the default tool auto-approval middleware is enabled because standing approval state is +session-scoped. + +## Capability defaults + +| Capability | Current behavior | +| --- | --- | +| Function invocation and per-service-call history persistence | Enabled by the harness | +| Todo provider and plan/execute mode | Enabled unless disabled or replaced | +| File memory | Enabled by default; backed by a filesystem store unless replaced | +| File access | Opt-in through `file_access_store`; experimental | +| Tool auto-approval middleware | Enabled unless disabled; still respects tool approval requirements | +| Web search | Added only when the client supports it and it is not disabled | +| Compaction | Enabled when token budgets or custom compaction strategies are supplied | +| Skills | Opt-in through `skills_paths` or `skills_provider` | +| Background agents | Opt-in and experimental | +| Shell | Opt-in, prerelease, and requires `agent-framework-tools` | +| Looping | Opt-in and experimental; bound with `loop_max_iterations` | +| OpenTelemetry | Wired through the harness telemetry layer | + +Do not describe an optional capability as active unless its required option is present. + +## Compaction + +Supply both budgets for the default token-aware strategies: + +```python +agent = create_harness_agent( + client=client, + max_context_window_tokens=128_000, + max_output_tokens=16_384, + agent_instructions="Research carefully and cite reliable sources.", +) +``` + +Without token budgets or custom before/after strategies, compaction is disabled. + +## File memory and file access + +- File memory is session-scoped and enabled by default. Treat it as persisted application data. +- File access is separate and opt-in through an `AgentFileStore`, commonly `FileSystemAgentFileStore`. +- File access tools require approval by default. +- Prefer `FileAccessProvider.read_only_tools_auto_approval_rule` when reads may run automatically but writes must + remain approval-gated. +- Use `file_access_disable_write_tools=True` when the application must be read-only. +- Keep stores rooted in an intentional application directory. + +Example source: +[Harness data processing](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/harness/harness_data_processing.py). + +## Skills + +Use `skills_paths` for file-based Agent Skills and `skills_provider` for custom or code-defined skills. When neither is +provided, no `SkillsProvider` is added. + +- Load skills only from trusted locations. +- Treat externally supplied skill content as untrusted input. +- Keep skill loading, resource reads, and script execution approval-gated unless the application intentionally changes + those policies. + +## Looping and background work + +Looping is opt-in: + +```python +from agent_framework import create_harness_agent, todos_remaining, todos_remaining_message + +agent = create_harness_agent( + client=client, + loop_should_continue=todos_remaining(looping_modes=["execute"]), + loop_next_message=todos_remaining_message, + loop_max_iterations=10, +) +``` + +- Always set a reasonable iteration cap. +- Stop for pending approvals or user input. +- Give background agents unique names, narrow responsibilities, and explicit result contracts. +- Treat background-agent input and output as potential prompt-injection and data-exfiltration boundaries. + +Looping and background agents currently emit experimental warnings. + +## Shell integration + +Pass `shell_executor` only when command execution is an explicit product requirement. + +- Install the prerelease `agent-framework-tools` distribution for the supplied shell implementations. +- Confine the working directory and environment. +- Keep command execution approval-gated. +- Treat command filters as a usability safeguard, not a security boundary. +- The caller owns the executor lifecycle. + +## Validation + +1. Construct the harness and create a session. +2. Run a trivial prompt. +3. Exercise todos and modes if enabled. +4. Verify file memory and file-access roots. +5. Verify approval behavior for a side-effecting tool. +6. Prove loop termination and iteration limits when looping is enabled. +7. Clean up shell and background-agent resources. + +Current examples: + +- [First harness](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/06_agent_harness.py) +- [Research harness](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/harness/harness_research.py) diff --git a/skills/agent-framework-builder-python/references/hosting-and-deployment.md b/skills/agent-framework-builder-python/references/hosting-and-deployment.md new file mode 100644 index 00000000000..051175f205e --- /dev/null +++ b/skills/agent-framework-builder-python/references/hosting-and-deployment.md @@ -0,0 +1,150 @@ +# Hosting and deployment + +Use this path after the agent or workflow works locally. Keep Agent Framework application readiness separate from +cloud provisioning, RBAC, quota, networking, and deployment operations. + +Hosting packages have independent lifecycle states. Verify current package maturity and host contracts before +generating files. + +## Choose the hosting family + +| Need | Hosting family | +| --- | --- | +| Local execution | Direct Python process | +| Protocol hosting in any self-managed environment | General `agent-framework-hosting*` packages | +| Microsoft Foundry Agent Server | `agent-framework-foundry-hosting` | +| Durable execution or Azure Functions | Durable Agent Framework extension | + +Start with [hosting guidance](https://learn.microsoft.com/agent-framework/get-started/hosting) and then open the +selected host's detailed documentation and current sample. + +## General hosting + +The general hosting packages expose Agent Framework applications through protocols without targeting a specific cloud +or deployment environment. The application owns the process, network endpoint, scaling, identity, persistence, and +deployment platform. + +| Protocol or surface | Package | +| --- | --- | +| Shared hosting abstractions | `agent-framework-hosting` | +| OpenAI-compatible Responses | `agent-framework-hosting-responses` | +| Agent-to-Agent (A2A) | `agent-framework-hosting-a2a` | +| Model Context Protocol (MCP) | `agent-framework-hosting-mcp` | +| Telegram | `agent-framework-hosting-telegram` | + +These packages provide host adapters and protocol behavior; they do not provision or deploy the surrounding +infrastructure. They are currently alpha packages, so verify their current contracts before depending on them. + +Treat protocol request bodies and translated run options as caller-controlled input. Forward only an explicit allowlist +of options that the application supports; do not pass caller-supplied tools, instructions, or +`additional_function_arguments` into `agent.run(...)` unless the application deliberately exposes those capabilities. + +## Foundry hosting + +`agent-framework-foundry-hosting` is specifically for the Microsoft Foundry Agent Server and currently provides at +least two hosting protocols: + +| Foundry protocol | API | +| --- | --- | +| OpenAI-compatible Responses | `ResponsesHostServer` | +| Custom invocation request and response contracts | `InvocationsHostServer` | + +This package integrates Agent Framework sessions, responses, approvals, workflows, and Foundry sandbox identity with +the platform host. It is not the general-purpose hosting layer for arbitrary environments. + +### Responses protocol + +`agent-framework-foundry-hosting` is currently beta, so allow prerelease resolution when installing it. + +```python +from __future__ import annotations + +import os + +from agent_framework import Agent +from agent_framework.foundry import FoundryChatClient, ResponsesHostServer +from azure.identity import DefaultAzureCredential + + +agent = Agent( + client=FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + credential=DefaultAzureCredential(), + ), + instructions="You are a helpful assistant.", + default_options={"store": False}, +) + +ResponsesHostServer(agent=agent, history_source="agent_server").run() +``` + +Choose the history source deliberately: + +- `agent_server` lets the host reconstruct outer Responses history and disables duplicate inner storage; +- `service` uses the downstream service's private continuation; +- `agent` lets the agent's history provider or client storage own history. + +Do not combine competing history owners. + +### Invocations protocol + +Use `InvocationsHostServer` when the hosted application needs its own invocation payload and response contract instead +of the OpenAI Responses protocol. Supply the documented parser or invocation handler and make session, identity, and +workflow continuation behavior explicit. + +## Foundry agent and workflow ownership + +- Passing an agent instance reuses it for the host lifetime. +- Pass a zero-argument sync or async factory when the agent owns mutable state outside `AgentSession`. +- Prefer native `workflow=` hosting when the host supports workflows. +- Hosting `workflow.as_agent()` is deprecated in Foundry hosting because a shared workflow agent is stateful and can + leak state across requests. +- Workflow factories must build fresh workflows, executors, agents, clients, providers, and mutable tools while + preserving stable workflow and executor identities for checkpoint restoration. + +## Entry-point contracts + +- `build_agent()` is an application convention, not a framework requirement. +- Export a module-level `agent` only when the selected discovery or hosting path requires it. +- Foundry host servers accept an agent instance or factory through `agent=`. +- Native Foundry workflow hosting has its own `workflow=` and request-parsing contracts. +- General Responses, A2A, MCP, Telegram, and durable hosts use their own adapters; follow the selected package's + examples instead of applying Foundry-specific server contracts. + +## Configuration and identity + +Document only variables required by the selected provider and host. Common Foundry examples use: + +- `FOUNDRY_PROJECT_ENDPOINT`; +- `FOUNDRY_MODEL`; +- a development credential such as `AzureCliCredential`; +- a deliberate production credential, commonly managed identity. + +Do not commit secrets. Review `DefaultAzureCredential` for production because its fallback chain may probe credentials +the application did not intend to use. + +## Deployment readiness checklist + +- `pyproject.toml` declares only required runtime packages. +- The selected entry point imports and constructs the agent or workflow. +- Required environment variables fail with clear messages. +- Session, checkpoint, response, and service-history ownership are explicit. +- Tool side effects and approval behavior are documented. +- External files, skills, prompts, and resources are packaged intentionally. +- Health, logging, telemetry, and sensitive-data settings match the host. +- Request-scoped resources are created and cleaned up by the request-scoped factory. +- The deployment package contains no local credentials, caches, or unrelated files. +- Host-specific manifests are added only by the deployment workflow that consumes them. + +## Foundry deployment handoff + +After local readiness is proven, use dedicated Foundry tooling for provisioning, deployment, invocation, traces, RBAC, +quota, capacity, and troubleshooting. Do not invent `azure.yaml`, manifests, or `.foundry/` layouts without the +deployment workflow that consumes them. + +Current examples: + +- [First hosted agent](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/07_hosting.py) +- [Foundry hosting package guidance](https://github.com/microsoft/agent-framework/blob/main/python/packages/foundry_hosting/README.md) +- [Foundry hosted-agent samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/04-hosting/foundry-hosted-agents) diff --git a/skills/agent-framework-builder-python/references/integrations.md b/skills/agent-framework-builder-python/references/integrations.md new file mode 100644 index 00000000000..5f73eacea49 --- /dev/null +++ b/skills/agent-framework-builder-python/references/integrations.md @@ -0,0 +1,129 @@ +# Providers and integrations + +Choose integrations only after the application contract is clear. A model provider or agent service supplies the +underlying intelligence; tools, memory, protocols, UI, and hosting add separate capabilities. + +Package and provider availability changes more frequently than core agent patterns. Check current Microsoft Learn +documentation and package metadata before adding dependencies or promising feature parity. + +## Python version compatibility + +Agent Framework supports Python 3.10 through 3.15. Python 3.10 remains supported and tested for the current release but +is deprecated because upstream Python 3.10 has reached end of life. New environments should use Python 3.11 or later. + +Python 3.15 supports the core framework and most integrations, with these current limitations: + +| Area | Python 3.15 limitation | +| --- | --- | +| Declarative Power Fx | `agent-framework-declarative` installs without its `powerfx` dependency. Expressions beginning with `=` are not evaluated. | +| Redis | `agent-framework-redis` is unavailable because RedisVL requires Python earlier than 3.15. | +| Hyperlight | `agent-framework-hyperlight` is unavailable until a compatible sandbox backend wheel is published. | +| AG-UI A2UI | Base `agent-framework-ag-ui` works, but its optional `a2ui` extra is unavailable because the toolkit requires Python earlier than 3.15. | + +Check each selected distribution's `Requires-Python` metadata instead of assuming every integration supports the full +framework range. + +## Providers and agent services + +Common released distributions include: + +- `agent-framework-foundry` for Microsoft Foundry chat clients, managed agents, memory, embeddings, and Foundry evals; +- `agent-framework-openai` for OpenAI and Azure OpenAI Responses or Chat Completions clients; +- `agent-framework-github-copilot` for GitHub Copilot-backed scenarios. + +Other providers and agent services have independent lifecycle states and Python requirements. Install only the selected +distribution and allow prerelease packages only when required. + +Start with the current +[model provider catalog](https://learn.microsoft.com/agent-framework/integrations/by-component/model-providers/) and +[agent service catalog](https://learn.microsoft.com/agent-framework/integrations/by-component/agent-services/). + +Verify: + +- authentication and identity; +- Python and package version support; +- streaming and structured output; +- hosted tools; +- application-managed versus service-managed history; +- tool approval and function-loop behavior. + +## Tools + +Choose local versus hosted execution deliberately: + +- function tools for application-owned Python functions; +- local MCP for caller-owned MCP clients and sessions; +- hosted MCP, web search, file search, or code interpreter only when the selected provider supports them; +- Agent Skills through `SkillsProvider` when the built agent should discover reusable capabilities. + +Keep local function and MCP side effects under the application's approval and security policy. Hosted execution still +requires explicit identity, data-boundary, and provider-capability review. + +## Memory, retrieval, and vector stores + +Use normal sessions and context providers first. Add a connector when persistence, scale, search, or an external data +model requires it. + +Current integration categories include: + +- chat history and context providers; +- Foundry memory; +- Azure AI Search; +- Cosmos DB and Redis; +- vector-store connectors such as PostgreSQL, MongoDB, Qdrant, DuckDB, Oracle, SQL Server, and DocumentDB; +- graph and third-party memory providers. + +Some connector packages are alpha or beta even when core is released. Check lifecycle state and supported operations. + +Define tenant, user, retention, deletion, authorization, and encryption boundaries before persisting conversation, +memory, or vector data. + +## Protocol and UI integrations + +- A2A for interoperable agent-to-agent communication. +- MCP hosting when the application exposes tools through MCP. +- ChatKit and other application-specific UI integrations where appropriate. + +Protocol integration does not replace authorization. Validate identities, payloads, exposed tools, and cross-tenant +behavior at every boundary. + +### AG-UI + +Use `agent-framework-ag-ui` to connect Agent Framework agents and workflows to AG-UI-compatible frontends. + +- `add_agent_framework_fastapi_endpoint(...)` exposes a `SupportsAgentRun` implementation or `Workflow` through a + FastAPI endpoint. +- `AgentFrameworkAgent` and `AgentFrameworkWorkflow` provide the protocol adapters. +- `AGUIChatClient` connects an Agent Framework application to an existing AG-UI server. +- AG-UI supports streaming events, tool calls, shared state, interrupts, approval resume, and thread snapshots. +- Use a thread-scoped workflow factory when workflow instances hold mutable runtime state. +- AG-UI thread and run IDs are correlation identifiers, not authentication or tenant-isolation boundaries. +- The application owns authorization, durable snapshot storage, scaling, and frontend rendering policy. + +Current starting point: +[AG-UI single-agent sample](https://github.com/microsoft/agent-framework/tree/main/python/samples/05-end-to-end/ag_ui_single_agent). + +## Durable and cloud integrations + +Durable Task and Azure Functions integrations are maintained in the +[Durable Agent Framework extension](https://github.com/microsoft/agent-framework-durable-extension), not in the main +Python package tree. + +Foundry hosting and other host adapters have separate packages and lifecycle states. Read +`hosting-and-deployment.md` before adding host or deployment files. + +## Selection checklist + +1. Confirm the provider or service and authentication model. +2. Check package lifecycle and Python support. +3. Check required provider features rather than assuming parity. +4. Add only the selected integration packages. +5. Define data, network, identity, approval, and retention boundaries. +6. Build a focused integration test or opt-in live smoke test. +7. Document required environment variables and permissions. + +Current catalogs: + +- [Agent Framework integrations](https://learn.microsoft.com/agent-framework/integrations/) +- [Python package status](https://github.com/microsoft/agent-framework/blob/main/python/PACKAGE_STATUS.md) +- [Python samples](https://github.com/microsoft/agent-framework/tree/main/python/samples) diff --git a/skills/agent-framework-builder-python/references/observability.md b/skills/agent-framework-builder-python/references/observability.md new file mode 100644 index 00000000000..c4f32ee135a --- /dev/null +++ b/skills/agent-framework-builder-python/references/observability.md @@ -0,0 +1,43 @@ +# Observability + +Use observability to diagnose and monitor agent runs, model calls, tool calls, workflow execution, latency, errors, and +token use. Observability provides runtime evidence; it does not define whether an output is correct. + +## OpenTelemetry configuration + +Agent Framework instrumentation is enabled by default, but the application must configure OpenTelemetry providers and +exporters. The helper below configures them from arguments and standard environment variables: + +```python +from agent_framework.observability import configure_otel_providers + + +configure_otel_providers(enable_sensitive_data=False) +``` + +- Call `configure_otel_providers(...)` once during application startup. +- Keep sensitive prompts, tool arguments, tool results, and model responses disabled by default. +- Use `enable_instrumentation(...)` only to control instrumentation when providers are configured elsewhere. +- Use traces to correlate agent runs, model calls, tools, MCP calls, and workflow executors. +- Give agents, harnesses, workflows, and services stable identifiers. +- When sensitive data is disabled, built-in Agent Framework telemetry redacts exception details from spans, events, + workflow telemetry, and failure logs, and does not attach serialized tool arguments to invocation metrics. +- Apply equivalent redaction and cardinality controls to application-defined telemetry and custom exporters. + +## What to inspect + +- Agent and workflow run boundaries. +- Model requests, responses, latency, and token usage. +- Function and MCP tool calls, including failures and approval pauses. +- Workflow executor transitions, checkpoint activity, and suspended runs. +- Provider errors, retries, and hosted-service continuations. + +Avoid using telemetry as a storage mechanism for application state or sensitive conversation history. + +See [agent observability](https://learn.microsoft.com/agent-framework/agents/observability) and +[workflow observability](https://learn.microsoft.com/agent-framework/workflows/observability). + +Current examples: + +- [Agent observability](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/observability/agent_observability.py) +- [Workflow observability](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/observability/workflow_observability.py) diff --git a/skills/agent-framework-builder-python/references/workflow-builder.md b/skills/agent-framework-builder-python/references/workflow-builder.md new file mode 100644 index 00000000000..f139745617e --- /dev/null +++ b/skills/agent-framework-builder-python/references/workflow-builder.md @@ -0,0 +1,143 @@ +# Workflow and orchestration builder + +Use this path when explicit control flow, multiple participants, resumable progress, or caller-visible intermediate work +matters. + +## Selection + +| Need | API and package | +| --- | --- | +| Simple Pythonic async composition | Experimental `@workflow` from `agent-framework-core` | +| Cached calls, step events, or step-level checkpoint replay | Experimental `@step` inside `@workflow` | +| Explicit typed topology, edges, executors, routing, and outputs | `WorkflowBuilder` from `agent-framework-core` | +| Standard multi-agent collaboration pattern | Builder from released `agent-framework-orchestrations` | +| YAML-defined actions and routing | `WorkflowFactory` from `agent-framework-declarative`; read `declarative.md` | + +Prefer a plain agent or harness when an inspectable execution path is not part of the requirement. + +## Functional workflow + +`@workflow` and `@step` are experimental. The decorator creates a stateless definition; call `build()` to create a +stateful workflow for one logical caller or session. + +```python +from agent_framework import workflow + + +async def normalize(text: str) -> str: + return text.strip().upper() + + +@workflow +async def text_workflow(text: str) -> str: + return await normalize(text) + + +result = await text_workflow.build().run(" hello ") +print(result.get_outputs()[0]) +``` + +Normal async functions work inside a functional workflow. Add `@step` only where caching, checkpoint replay, event +emission, or step-level observability justify it. + +## Graph workflow + +Use executors and edges when explicit routing and typed data flow matter: + +```python +from agent_framework import WorkflowBuilder, WorkflowContext, executor +from typing_extensions import Never + + +@executor(id="normalize") +async def normalize(text: str, ctx: WorkflowContext[str]) -> None: + await ctx.send_message(text.strip().upper()) + + +@executor(id="finish") +async def finish(text: str, ctx: WorkflowContext[Never, str]) -> None: + await ctx.yield_output(text) + + +workflow = WorkflowBuilder(start_executor=normalize).add_edge(normalize, finish).build() +result = await workflow.run(" hello ") +``` + +Build a fresh workflow when executors or wrapped agents hold mutable run-specific state. + +## Agents in workflows + +Agents can be workflow executors: + +```python +workflow = WorkflowBuilder(start_executor=writer).add_edge(writer, reviewer).build() +result = await workflow.run("Draft a product announcement.") +``` + +Decide whether each participant needs an independent session, service-managed history, tools, middleware, or context. +Do not assume one agent's session is valid for another. + +## Standard orchestrations + +Install the released orchestration package when using these builders: + +```bash +uv add agent-framework-orchestrations +``` + +Import from `agent_framework.orchestrations`: + +- `SequentialBuilder`: staged output from one participant to the next. +- `ConcurrentBuilder`: fan-out to participants and aggregate. +- `HandoffBuilder`: transfer control among configured specialists. +- `GroupChatBuilder`: managed multi-participant conversation. +- `MagenticBuilder`: manager/planner-driven collaboration. + +Use the standard builder before implementing an equivalent custom graph. + +## Output selection + +Configure `output_from` for caller-facing workflow output and `intermediate_output_from` for progress or supporting +outputs. Without explicit selection, output-capable executors use the workflow's documented default behavior. + +When exposing a workflow through `workflow.as_agent()`, verify that its start executor accepts agent message input and +that callers actually require the `Agent` contract. Prefer native workflow hosting where the host supports it. + +## Agent-to-workflow migration + +Converting one agent into a workflow changes contracts, state, outputs, and often deployment behavior: + +1. Record the current input, response, tools, middleware, context, session, and hosting contract. +2. Choose a functional workflow, graph workflow, or standard orchestration. +3. Split responsibilities only where the split changes behavior or observability. +4. Define data passed between stages. +5. Select final and intermediate outputs. +6. Preserve session and history behavior intentionally. +7. Revisit tool approval across every participant and branch. +8. Keep an agent adapter only when the external contract requires it. + +## Gotchas + +- Functional workflow definitions must be built before running. +- A built workflow is stateful and cannot run concurrently. +- Handoff relationships must be configured explicitly. +- Magentic participants need stable, unique names. +- Checkpoint resume requires stable workflow, executor, and participant identities. +- Human approval of a tool call and human input at a workflow transition are different mechanisms. +- Workflow streams include framework events in addition to application output. + +## Validation + +1. Run the smallest graph with deterministic executors before adding model calls. +2. Assert final and intermediate outputs separately. +3. Exercise every route or branch. +4. Test suspend/resume when human input or checkpoints are used. +5. Smoke-test streaming event types when callers consume streams. +6. Verify participant sessions, approval behavior, and error propagation. +7. Inspect a visualization when topology is non-trivial. + +Current examples: + +- [Functional workflow](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/05a_functional_workflow_basics.py) +- [Graph workflow](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/05c_first_graph_workflow.py) +- [Agents in a workflow](https://github.com/microsoft/agent-framework/blob/main/python/samples/03-workflows/_start-here/step2_agents_in_a_workflow.py) From 8c60d016d26673228d7f2bd23b9f4a0e862dac2d Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Wed, 14 Oct 2026 17:19:02 +0200 Subject: [PATCH 02/10] Address consumer skill review feedback --- .../references/harness-agent-builder.md | 4 ++++ .../references/integrations.md | 18 +++++++++--------- .../references/observability.md | 9 ++++++++- 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/skills/agent-framework-builder-python/references/harness-agent-builder.md b/skills/agent-framework-builder-python/references/harness-agent-builder.md index 5b405c49504..e8c567b1976 100644 --- a/skills/agent-framework-builder-python/references/harness-agent-builder.md +++ b/skills/agent-framework-builder-python/references/harness-agent-builder.md @@ -109,6 +109,10 @@ Without token budgets or custom before/after strategies, compaction is disabled. remain approval-gated. - Use `file_access_disable_write_tools=True` when the application must be read-only. - Keep stores rooted in an intentional application directory. +- For hosted or untrusted workloads, do not use the shared default `{cwd}/agent-file-memory` store without operational + controls. Either disable file memory or supply tenant-scoped storage with enforced byte and file-count quotas, + retention limits, and server-owned cleanup. File-memory writes do not require tool approval, so storage isolation + and resource limits must be enforced by the host. Example source: [Harness data processing](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/harness/harness_data_processing.py). diff --git a/skills/agent-framework-builder-python/references/integrations.md b/skills/agent-framework-builder-python/references/integrations.md index 5f73eacea49..8b5030adbcc 100644 --- a/skills/agent-framework-builder-python/references/integrations.md +++ b/skills/agent-framework-builder-python/references/integrations.md @@ -8,17 +8,17 @@ documentation and package metadata before adding dependencies or promising featu ## Python version compatibility -Agent Framework supports Python 3.10 through 3.15. Python 3.10 remains supported and tested for the current release but -is deprecated because upstream Python 3.10 has reached end of life. New environments should use Python 3.11 or later. +Agent Framework supports Python 3.11 through 3.15. Build new environments with the latest supported Python release, or +at least the previous release (N-1), unless a required integration has a narrower compatibility range. -Python 3.15 supports the core framework and most integrations, with these current limitations: +Python 3.14 and 3.15 support the core framework, with these current integration limitations: -| Area | Python 3.15 limitation | -| --- | --- | -| Declarative Power Fx | `agent-framework-declarative` installs without its `powerfx` dependency. Expressions beginning with `=` are not evaluated. | -| Redis | `agent-framework-redis` is unavailable because RedisVL requires Python earlier than 3.15. | -| Hyperlight | `agent-framework-hyperlight` is unavailable until a compatible sandbox backend wheel is published. | -| AG-UI A2UI | Base `agent-framework-ag-ui` works, but its optional `a2ui` extra is unavailable because the toolkit requires Python earlier than 3.15. | +| Area | Affected versions | Limitation | +| --- | --- | --- | +| Declarative Power Fx | Python 3.14-3.15 | `agent-framework-declarative` installs without its `powerfx` dependency. Expressions beginning with `=` are not evaluated. | +| Redis | Python 3.15 | `agent-framework-redis` is unavailable because RedisVL requires Python earlier than 3.15. | +| Hyperlight | Python 3.15 | `agent-framework-hyperlight` is unavailable until a compatible sandbox backend wheel is published. | +| AG-UI A2UI | Python 3.15 | Base `agent-framework-ag-ui` works, but its optional `a2ui` extra is unavailable because the toolkit requires Python earlier than 3.15. | Check each selected distribution's `Requires-Python` metadata instead of assuming every integration supports the full framework range. diff --git a/skills/agent-framework-builder-python/references/observability.md b/skills/agent-framework-builder-python/references/observability.md index c4f32ee135a..167846223cf 100644 --- a/skills/agent-framework-builder-python/references/observability.md +++ b/skills/agent-framework-builder-python/references/observability.md @@ -6,7 +6,14 @@ token use. Observability provides runtime evidence; it does not define whether a ## OpenTelemetry configuration Agent Framework instrumentation is enabled by default, but the application must configure OpenTelemetry providers and -exporters. The helper below configures them from arguments and standard environment variables: +exporters. Narrow Agent Framework installations include `opentelemetry-api` but not the SDK. Install +`opentelemetry-sdk` and the exporter package required by the application's telemetry backend before using the helper: + +```bash +python -m pip install opentelemetry-sdk opentelemetry-exporter-otlp +``` + +The helper configures providers from arguments and standard environment variables: ```python from agent_framework.observability import configure_otel_providers From 72de2327244d978048b6e7bc5b8988c1106fb6aa Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Wed, 14 Oct 2026 19:06:09 +0200 Subject: [PATCH 03/10] Add Agent Skills validation workflow --- .github/workflows/agent-skills.yml | 72 ++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 .github/workflows/agent-skills.yml diff --git a/.github/workflows/agent-skills.yml b/.github/workflows/agent-skills.yml new file mode 100644 index 00000000000..bffe7ea8c51 --- /dev/null +++ b/.github/workflows/agent-skills.yml @@ -0,0 +1,72 @@ +name: Validate Agent Skills + +on: + workflow_dispatch: + pull_request: + paths: + - "skills/**" + - ".github/workflows/agent-skills.yml" + push: + branches: ["main"] + paths: + - "skills/**" + - ".github/workflows/agent-skills.yml" + +permissions: + contents: read + +jobs: + validate: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.13" + + - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0 + with: + version-file: "python/pyproject.toml" + + - name: Validate Agent Skills specification + shell: bash + run: | + set -euo pipefail + + found_skill=0 + while IFS= read -r skill_file; do + [[ -z "$skill_file" ]] && continue + found_skill=1 + uvx --from "skills-ref==0.1.1" agentskills validate "$(dirname "$skill_file")" + done < <(find skills -mindepth 2 -maxdepth 2 -name SKILL.md -print | sort) + + if (( found_skill == 0 )); then + echo "No public Agent Skills found under skills/." >&2 + exit 1 + fi + + - name: Validate relative resource links + run: | + python - <<'PY' + from pathlib import Path + import re + + missing: list[str] = [] + for path in Path("skills").rglob("*.md"): + for target in re.findall(r"\[[^\]]+\]\(([^)]+)\)", path.read_text(encoding="utf-8")): + if "://" in target or target.startswith(("#", "mailto:")): + continue + relative_path = target.split("#", 1)[0] + if relative_path and not (path.parent / relative_path).resolve().exists(): + missing.append(f"{path}: {target}") + + if missing: + raise SystemExit("\n".join(missing)) + PY + + - name: Validate GitHub skill publishing + env: + GH_TOKEN: ${{ github.token }} + run: gh skill publish skills --dry-run From 24dc34125dc853e530ad50061d8456461a37690d Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Wed, 14 Oct 2026 20:14:36 +0200 Subject: [PATCH 04/10] Expand skill extension and deployment guidance --- .../references/agent-builder.md | 69 +++++++++++++++++-- .../references/hosting-and-deployment.md | 30 ++++++-- .../references/integrations.md | 4 ++ 3 files changed, 93 insertions(+), 10 deletions(-) diff --git a/skills/agent-framework-builder-python/references/agent-builder.md b/skills/agent-framework-builder-python/references/agent-builder.md index 1036862fd9a..a7604097993 100644 --- a/skills/agent-framework-builder-python/references/agent-builder.md +++ b/skills/agent-framework-builder-python/references/agent-builder.md @@ -90,6 +90,44 @@ tools for those operations. Use current provider documentation before assuming hosted-tool, storage, streaming, or structured-output parity. +## Build a custom provider + +Build a provider when the application must integrate a model endpoint or remote agent service that no existing Agent +Framework package supports, or when a private gateway requires custom authentication, transport, option mapping, or +protocol translation. + +Choose the extension point that matches the backend: + +- Implement a chat client when the backend accepts messages and returns model-style chat responses. Subclass + `BaseChatClient` and implement `_inner_get_response(...)` for both regular and streaming execution. +- Implement an agent when the backend already owns agent behavior or does not fit chat-client semantics. Implement + `SupportsAgentRun`, usually by subclassing `BaseAgent`, and support both `stream=False` and `stream=True`. +- Use middleware instead of a provider when the underlying model protocol is already supported and the requirement is + cross-cutting policy, retries, routing, logging, or request transformation. + +A custom chat client is responsible for: + +- converting Agent Framework `Message` and `Content` values to the backend request; +- mapping backend responses, streaming updates, usage, identifiers, and finish information back to framework types; +- defining typed provider-specific options without discarding unknown application options unintentionally; +- mapping provider failures to actionable exceptions and preserving cancellation; +- declaring service-managed history behavior and persisting continuation identifiers through `AgentSession` when + supported; +- owning and cleaning up network clients or other context-managed resources it creates. + +If a custom client needs the standard tool loop, chat middleware, and telemetry, compose the public layers in this +order: function invocation, chat middleware, chat telemetry, then the raw or base client. Prefer an existing +fully-featured client when only configuration or middleware differs. + +Validate regular and streaming parity, tools, structured output, usage, cancellation, session continuation, resource +cleanup, and middleware ordering. + +Current references: + +- [BaseChatClient API](https://learn.microsoft.com/python/api/agent-framework-core/agent_framework.basechatclient?view=agent-framework-python-latest) +- [Custom chat client sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/chat_client/custom_chat_client.py) +- [Custom agent sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/providers/custom/custom_agent.py) + ## Tools and approval - Prefer focused, typed, non-interactive tool functions. @@ -150,13 +188,34 @@ dynamically for a run or session. ## Middleware -Choose the narrowest middleware layer: +Use middleware for reusable behavior that belongs around an execution boundary rather than in agent instructions, +business tools, or a provider implementation. Common cases include policy enforcement, retries, caching, routing, +dynamic tool exposure, validation, observability, result transformation, and shared runtime context. + +Choose the narrowest boundary: + +- **Agent middleware** runs around `Agent.run(...)`. Use it for run-level policy, caching, termination, result + overrides, or shared state. +- **Chat middleware** runs around each model call, including calls inside a tool loop. Use it for retries, request and + response transformation, model routing, or per-call usage behavior. +- **Function middleware** runs around each local tool invocation. Use it for validated-argument checks, authorization + context, logging, exception handling, or tool-result transformation. + +Create middleware either by subclassing `AgentMiddleware`, `ChatMiddleware`, or `FunctionMiddleware`, or by using the +matching decorator. The implementation receives a context and `call_next`: + +1. Inspect or update the context before execution. +2. Call `await call_next()` without passing the context again. +3. Inspect or transform the result after execution. + +Skip `call_next()` only when deliberately terminating the pipeline or returning a complete valid result. Keep state +scoped to the run or session unless cross-run sharing is intentional, and test both streaming and non-streaming paths. -- agent middleware for run-level policy, termination, result overrides, and shared state; -- chat middleware for model request and response behavior; -- function middleware for tool invocation behavior. +Current references: -In Python middleware, call `await call_next()` without passing the context again. +- [Middleware concepts](https://learn.microsoft.com/agent-framework/concepts/agents/middleware/) +- [Middleware samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/02-agents/middleware) +- [Class-based middleware sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/middleware/class_based_middleware.py) ## Validation diff --git a/skills/agent-framework-builder-python/references/hosting-and-deployment.md b/skills/agent-framework-builder-python/references/hosting-and-deployment.md index 051175f205e..725f7cd8699 100644 --- a/skills/agent-framework-builder-python/references/hosting-and-deployment.md +++ b/skills/agent-framework-builder-python/references/hosting-and-deployment.md @@ -137,11 +137,31 @@ the application did not intend to use. - The deployment package contains no local credentials, caches, or unrelated files. - Host-specific manifests are added only by the deployment workflow that consumes them. -## Foundry deployment handoff - -After local readiness is proven, use dedicated Foundry tooling for provisioning, deployment, invocation, traces, RBAC, -quota, capacity, and troubleshooting. Do not invent `azure.yaml`, manifests, or `.foundry/` layouts without the -deployment workflow that consumes them. +## Deploy a hosted agent to Microsoft Foundry + +After local readiness is proven, follow the Foundry Hosted Agent deployment flow rather than inventing manifests or +infrastructure files: + +1. Start from a compatible `agent.manifest.yaml` for the selected Responses or Invocations host. +2. Initialize the deployment project with `azd ai agent init -m `. +3. Run `azd provision` only when the required Foundry project, model deployment, registry, and monitoring resources do + not already exist. +4. Test the host locally with `azd ai agent run` and `azd ai agent invoke --local`. +5. Deploy with `azd deploy`. + +Use these instructions according to the scenario: + +- [Agent Framework: Foundry hosted agents and deployment](https://learn.microsoft.com/agent-framework/hosting/foundry-hosted-agent#deploying-to-foundry) +- [Foundry quickstart: deploy your first hosted agent with azd](https://learn.microsoft.com/azure/foundry/agents/quickstarts/quickstart-hosted-agent) +- [Foundry: deploy a hosted agent](https://learn.microsoft.com/azure/foundry/agents/how-to/deploy-hosted-agent) +- [Foundry: deploy a hosted agent from source code](https://learn.microsoft.com/azure/foundry/agents/how-to/deploy-hosted-agent-code) +- [Host Agent Framework agents as Foundry hosted agents](https://learn.microsoft.com/azure/foundry/how-to/develop/framework-hosted-agents#deploy) +- [Manage deployed hosted agents](https://learn.microsoft.com/azure/foundry/agents/how-to/manage-hosted-agent) +- [Agent Framework hosted-agent sample deployment instructions](https://github.com/microsoft/agent-framework/tree/main/python/samples/04-hosting/foundry-hosted-agents#deploying-the-agent-to-foundry) + +Use dedicated Foundry tooling for provisioning, deployment, invocation, traces, RBAC, quota, capacity, and +troubleshooting. Host manifests and runtime environment variables are deployment contracts; copy them from the current +guide or sample for the selected protocol instead of constructing them from memory. Current examples: diff --git a/skills/agent-framework-builder-python/references/integrations.md b/skills/agent-framework-builder-python/references/integrations.md index 8b5030adbcc..13270c02d58 100644 --- a/skills/agent-framework-builder-python/references/integrations.md +++ b/skills/agent-framework-builder-python/references/integrations.md @@ -34,6 +34,10 @@ Common released distributions include: Other providers and agent services have independent lifecycle states and Python requirements. Install only the selected distribution and allow prerelease packages only when required. +If no existing provider matches the backend, use the +[custom provider guidance](agent-builder.md#build-a-custom-provider) to implement a `BaseChatClient` or +`SupportsAgentRun` adapter instead of forking an unrelated provider. + Start with the current [model provider catalog](https://learn.microsoft.com/agent-framework/integrations/by-component/model-providers/) and [agent service catalog](https://learn.microsoft.com/agent-framework/integrations/by-component/agent-services/). From c175182dfe586e5e2c1b4456ea9ca96318bea756 Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Wed, 14 Oct 2026 20:16:57 +0200 Subject: [PATCH 05/10] Document custom ContextProvider guidance --- .../references/agent-builder.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/skills/agent-framework-builder-python/references/agent-builder.md b/skills/agent-framework-builder-python/references/agent-builder.md index a7604097993..2e565a1589f 100644 --- a/skills/agent-framework-builder-python/references/agent-builder.md +++ b/skills/agent-framework-builder-python/references/agent-builder.md @@ -186,6 +186,38 @@ dynamically for a run or session. - Define when the provider loads and persists data, and keep its session-state keys distinct from other providers. - Treat retrieved and persisted context as sensitive and enforce tenant and user boundaries outside the model. +### Build a custom ContextProvider + +Build a custom `ContextProvider` when an application needs reusable context engineering that runs with an agent, such +as retrieval, user or tenant memory, dynamic instructions, per-run tools, request-scoped data, or post-run extraction. +Use middleware instead when the requirement is primarily execution control around the agent, model, or tools. + +1. Subclass `ContextProvider` and give each provider instance a stable, unique `source_id`. +2. Override `before_run(...)` to add context through `SessionContext`: + - `extend_instructions(...)` for dynamic instructions; + - `extend_messages(...)` for retrieved or remembered messages; + - `extend_tools(...)` for tools available only to this invocation; + - `extend_middleware(...)` for invocation-scoped chat or function middleware. +3. Override `after_run(...)` to inspect the input and response, extract new information, or persist provider-owned + state. +4. Store mutable provider data in the supplied provider-scoped `state` dictionary. Use `session.state` directly only + when deliberate coordination across providers is required. +5. Register custom persisted classes with `register_state_type(...)` at module import time so restored sessions can + deserialize them before the application loads a session. + +Set `after_run_once_per_turn=True` when an `AgentLoopMiddleware` should defer `after_run` until the whole user turn is +complete. Context providers may add chat or function middleware, but not agent middleware. + +Test context attribution, multiple sessions, serialization and restoration, provider ordering, missing data, and +failure behavior. For cross-session memory, include origin session attribution and enforce authorization before +injecting content. + +Current references: + +- [Context Provider concepts](https://learn.microsoft.com/agent-framework/concepts/agents/conversations/context-providers) +- [ContextProvider API](https://learn.microsoft.com/python/api/agent-framework-core/agent_framework.contextprovider) +- [Custom ContextProvider sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/context_providers/simple_context_provider.py) + ## Middleware Use middleware for reusable behavior that belongs around an execution boundary rather than in agent instructions, From f830554c964681d37f789392d9bdba998bffb4e6 Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Wed, 14 Oct 2026 20:20:10 +0200 Subject: [PATCH 06/10] Add Agent Framework extensibility guide --- .../agent-framework-builder-python/SKILL.md | 1 + .../references/agent-builder.md | 102 +-------------- .../references/extensibility.md | 121 ++++++++++++++++++ .../references/integrations.md | 2 +- 4 files changed, 125 insertions(+), 101 deletions(-) create mode 100644 skills/agent-framework-builder-python/references/extensibility.md diff --git a/skills/agent-framework-builder-python/SKILL.md b/skills/agent-framework-builder-python/SKILL.md index 02a6183e322..f5c25591fd3 100644 --- a/skills/agent-framework-builder-python/SKILL.md +++ b/skills/agent-framework-builder-python/SKILL.md @@ -53,6 +53,7 @@ relying on remembered preview-era syntax. | Declarative definition | YAML or JSON should be the source of truth for a supported agent or workflow. | [Declarative definitions](references/declarative.md) | | Evaluation | Expected outputs, tool calls, failed runs, or quality regressions are the main concern. | [Evaluation](references/evaluation.md) | | Observability | The task is diagnosing or monitoring runtime behavior with traces, logs, and metrics. | [Observability](references/observability.md) | +| Extensibility | The application needs a custom provider, ContextProvider, or middleware component. | [Extensibility](references/extensibility.md) | | Integration | The task is choosing or wiring a provider, protocol, memory store, tool service, UI, or platform extension. | [Integrations](references/integrations.md) | | Hosting and deployment | Working Agent Framework code needs a process host, protocol endpoint, identity contract, or deployment shape. | [Hosting and deployment](references/hosting-and-deployment.md) | diff --git a/skills/agent-framework-builder-python/references/agent-builder.md b/skills/agent-framework-builder-python/references/agent-builder.md index 2e565a1589f..e4dab1d9f46 100644 --- a/skills/agent-framework-builder-python/references/agent-builder.md +++ b/skills/agent-framework-builder-python/references/agent-builder.md @@ -90,44 +90,6 @@ tools for those operations. Use current provider documentation before assuming hosted-tool, storage, streaming, or structured-output parity. -## Build a custom provider - -Build a provider when the application must integrate a model endpoint or remote agent service that no existing Agent -Framework package supports, or when a private gateway requires custom authentication, transport, option mapping, or -protocol translation. - -Choose the extension point that matches the backend: - -- Implement a chat client when the backend accepts messages and returns model-style chat responses. Subclass - `BaseChatClient` and implement `_inner_get_response(...)` for both regular and streaming execution. -- Implement an agent when the backend already owns agent behavior or does not fit chat-client semantics. Implement - `SupportsAgentRun`, usually by subclassing `BaseAgent`, and support both `stream=False` and `stream=True`. -- Use middleware instead of a provider when the underlying model protocol is already supported and the requirement is - cross-cutting policy, retries, routing, logging, or request transformation. - -A custom chat client is responsible for: - -- converting Agent Framework `Message` and `Content` values to the backend request; -- mapping backend responses, streaming updates, usage, identifiers, and finish information back to framework types; -- defining typed provider-specific options without discarding unknown application options unintentionally; -- mapping provider failures to actionable exceptions and preserving cancellation; -- declaring service-managed history behavior and persisting continuation identifiers through `AgentSession` when - supported; -- owning and cleaning up network clients or other context-managed resources it creates. - -If a custom client needs the standard tool loop, chat middleware, and telemetry, compose the public layers in this -order: function invocation, chat middleware, chat telemetry, then the raw or base client. Prefer an existing -fully-featured client when only configuration or middleware differs. - -Validate regular and streaming parity, tools, structured output, usage, cancellation, session continuation, resource -cleanup, and middleware ordering. - -Current references: - -- [BaseChatClient API](https://learn.microsoft.com/python/api/agent-framework-core/agent_framework.basechatclient?view=agent-framework-python-latest) -- [Custom chat client sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/chat_client/custom_chat_client.py) -- [Custom agent sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/providers/custom/custom_agent.py) - ## Tools and approval - Prefer focused, typed, non-interactive tool functions. @@ -186,68 +148,8 @@ dynamically for a run or session. - Define when the provider loads and persists data, and keep its session-state keys distinct from other providers. - Treat retrieved and persisted context as sensitive and enforce tenant and user boundaries outside the model. -### Build a custom ContextProvider - -Build a custom `ContextProvider` when an application needs reusable context engineering that runs with an agent, such -as retrieval, user or tenant memory, dynamic instructions, per-run tools, request-scoped data, or post-run extraction. -Use middleware instead when the requirement is primarily execution control around the agent, model, or tools. - -1. Subclass `ContextProvider` and give each provider instance a stable, unique `source_id`. -2. Override `before_run(...)` to add context through `SessionContext`: - - `extend_instructions(...)` for dynamic instructions; - - `extend_messages(...)` for retrieved or remembered messages; - - `extend_tools(...)` for tools available only to this invocation; - - `extend_middleware(...)` for invocation-scoped chat or function middleware. -3. Override `after_run(...)` to inspect the input and response, extract new information, or persist provider-owned - state. -4. Store mutable provider data in the supplied provider-scoped `state` dictionary. Use `session.state` directly only - when deliberate coordination across providers is required. -5. Register custom persisted classes with `register_state_type(...)` at module import time so restored sessions can - deserialize them before the application loads a session. - -Set `after_run_once_per_turn=True` when an `AgentLoopMiddleware` should defer `after_run` until the whole user turn is -complete. Context providers may add chat or function middleware, but not agent middleware. - -Test context attribution, multiple sessions, serialization and restoration, provider ordering, missing data, and -failure behavior. For cross-session memory, include origin session attribution and enforce authorization before -injecting content. - -Current references: - -- [Context Provider concepts](https://learn.microsoft.com/agent-framework/concepts/agents/conversations/context-providers) -- [ContextProvider API](https://learn.microsoft.com/python/api/agent-framework-core/agent_framework.contextprovider) -- [Custom ContextProvider sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/context_providers/simple_context_provider.py) - -## Middleware - -Use middleware for reusable behavior that belongs around an execution boundary rather than in agent instructions, -business tools, or a provider implementation. Common cases include policy enforcement, retries, caching, routing, -dynamic tool exposure, validation, observability, result transformation, and shared runtime context. - -Choose the narrowest boundary: - -- **Agent middleware** runs around `Agent.run(...)`. Use it for run-level policy, caching, termination, result - overrides, or shared state. -- **Chat middleware** runs around each model call, including calls inside a tool loop. Use it for retries, request and - response transformation, model routing, or per-call usage behavior. -- **Function middleware** runs around each local tool invocation. Use it for validated-argument checks, authorization - context, logging, exception handling, or tool-result transformation. - -Create middleware either by subclassing `AgentMiddleware`, `ChatMiddleware`, or `FunctionMiddleware`, or by using the -matching decorator. The implementation receives a context and `call_next`: - -1. Inspect or update the context before execution. -2. Call `await call_next()` without passing the context again. -3. Inspect or transform the result after execution. - -Skip `call_next()` only when deliberately terminating the pipeline or returning a complete valid result. Keep state -scoped to the run or session unless cross-run sharing is intentional, and test both streaming and non-streaming paths. - -Current references: - -- [Middleware concepts](https://learn.microsoft.com/agent-framework/concepts/agents/middleware/) -- [Middleware samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/02-agents/middleware) -- [Class-based middleware sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/middleware/class_based_middleware.py) +For custom providers, ContextProviders, and middleware, read +[Extensibility](extensibility.md). ## Validation diff --git a/skills/agent-framework-builder-python/references/extensibility.md b/skills/agent-framework-builder-python/references/extensibility.md new file mode 100644 index 00000000000..c1dcb169609 --- /dev/null +++ b/skills/agent-framework-builder-python/references/extensibility.md @@ -0,0 +1,121 @@ +# Extensibility + +Use this reference when an application needs framework behavior that is not provided by an existing model or agent +provider, ContextProvider, or middleware component. + +## Choose an extension point + +| Requirement | Extension point | +| --- | --- | +| Connect an unsupported model-style endpoint | Custom `BaseChatClient` | +| Connect a remote agent service or non-chat runtime | Custom `SupportsAgentRun`, usually a `BaseAgent` | +| Add retrieval, memory, dynamic instructions, per-run tools, or post-run extraction | Custom `ContextProvider` | +| Add cross-cutting execution policy, retries, caching, validation, or transforms | Agent, chat, or function middleware | + +Prefer composition through these public extension points over forking an existing provider or modifying framework +internals. + +## Custom providers + +Build a provider when the application must integrate a model endpoint or remote agent service that no existing Agent +Framework package supports, or when a private gateway requires custom authentication, transport, option mapping, or +protocol translation. + +Choose the provider shape that matches the backend: + +- Implement a chat client when the backend accepts messages and returns model-style chat responses. Subclass + `BaseChatClient` and implement `_inner_get_response(...)` for both regular and streaming execution. +- Implement an agent when the backend already owns agent behavior or does not fit chat-client semantics. Implement + `SupportsAgentRun`, usually by subclassing `BaseAgent`, and support both `stream=False` and `stream=True`. +- Use middleware instead of a provider when the underlying model protocol is already supported and the requirement is + cross-cutting policy, retries, routing, logging, or request transformation. + +A custom chat client is responsible for: + +- converting Agent Framework `Message` and `Content` values to the backend request; +- mapping backend responses, streaming updates, usage, identifiers, and finish information back to framework types; +- defining typed provider-specific options without discarding unknown application options unintentionally; +- mapping provider failures to actionable exceptions and preserving cancellation; +- declaring service-managed history behavior and persisting continuation identifiers through `AgentSession` when + supported; +- owning and cleaning up network clients or other context-managed resources it creates. + +If a custom client needs the standard tool loop, chat middleware, and telemetry, compose the public layers in this +order: function invocation, chat middleware, chat telemetry, then the raw or base client. Prefer an existing +fully-featured client when only configuration or middleware differs. + +Current references: + +- [BaseChatClient API](https://learn.microsoft.com/python/api/agent-framework-core/agent_framework.basechatclient?view=agent-framework-python-latest) +- [Custom chat client sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/chat_client/custom_chat_client.py) +- [Custom agent sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/providers/custom/custom_agent.py) + +## Custom ContextProviders + +Build a custom `ContextProvider` when an application needs reusable context engineering that runs with an agent, such +as retrieval, user or tenant memory, dynamic instructions, per-run tools, request-scoped data, or post-run extraction. +Use middleware instead when the requirement is primarily execution control around the agent, model, or tools. + +1. Subclass `ContextProvider` and give each provider instance a stable, unique `source_id`. +2. Override `before_run(...)` to add context through `SessionContext`: + - `extend_instructions(...)` for dynamic instructions; + - `extend_messages(...)` for retrieved or remembered messages; + - `extend_tools(...)` for tools available only to this invocation; + - `extend_middleware(...)` for invocation-scoped chat or function middleware. +3. Override `after_run(...)` to inspect the input and response, extract new information, or persist provider-owned + state. +4. Store mutable provider data in the supplied provider-scoped `state` dictionary. Use `session.state` directly only + when deliberate coordination across providers is required. +5. Register custom persisted classes with `register_state_type(...)` at module import time so restored sessions can + deserialize them before the application loads a session. + +Set `after_run_once_per_turn=True` when an `AgentLoopMiddleware` should defer `after_run` until the whole user turn is +complete. Context providers may add chat or function middleware, but not agent middleware. + +For cross-session memory, include origin session attribution and enforce authorization before injecting content. + +Current references: + +- [Context Provider concepts](https://learn.microsoft.com/agent-framework/concepts/agents/conversations/context-providers) +- [ContextProvider API](https://learn.microsoft.com/python/api/agent-framework-core/agent_framework.contextprovider) +- [Custom ContextProvider sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/context_providers/simple_context_provider.py) + +## Custom middleware + +Use middleware for reusable behavior that belongs around an execution boundary rather than in agent instructions, +business tools, or a provider implementation. Common cases include policy enforcement, retries, caching, routing, +dynamic tool exposure, validation, observability, result transformation, and shared runtime context. + +Choose the narrowest boundary: + +- **Agent middleware** runs around `Agent.run(...)`. Use it for run-level policy, caching, termination, result + overrides, or shared state. +- **Chat middleware** runs around each model call, including calls inside a tool loop. Use it for retries, request and + response transformation, model routing, or per-call usage behavior. +- **Function middleware** runs around each local tool invocation. Use it for validated-argument checks, authorization + context, logging, exception handling, or tool-result transformation. + +Create middleware either by subclassing `AgentMiddleware`, `ChatMiddleware`, or `FunctionMiddleware`, or by using the +matching decorator. The implementation receives a context and `call_next`: + +1. Inspect or update the context before execution. +2. Call `await call_next()` without passing the context again. +3. Inspect or transform the result after execution. + +Skip `call_next()` only when deliberately terminating the pipeline or returning a complete valid result. Keep state +scoped to the run or session unless cross-run sharing is intentional, and test both streaming and non-streaming paths. + +Current references: + +- [Middleware concepts](https://learn.microsoft.com/agent-framework/concepts/agents/middleware/) +- [Middleware samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/02-agents/middleware) +- [Class-based middleware sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/middleware/class_based_middleware.py) + +## Validation + +- Custom providers: test regular and streaming parity, tools, structured output, usage, cancellation, session + continuation, resource cleanup, and layer ordering. +- Custom ContextProviders: test attribution, multiple sessions, serialization and restoration, provider ordering, + missing data, cross-session authorization, and failure behavior. +- Middleware: test the before/after paths, short-circuit behavior, exceptions, state scope, and both streaming and + non-streaming execution. diff --git a/skills/agent-framework-builder-python/references/integrations.md b/skills/agent-framework-builder-python/references/integrations.md index 13270c02d58..af3507ab23e 100644 --- a/skills/agent-framework-builder-python/references/integrations.md +++ b/skills/agent-framework-builder-python/references/integrations.md @@ -35,7 +35,7 @@ Other providers and agent services have independent lifecycle states and Python distribution and allow prerelease packages only when required. If no existing provider matches the backend, use the -[custom provider guidance](agent-builder.md#build-a-custom-provider) to implement a `BaseChatClient` or +[custom provider guidance](extensibility.md#custom-providers) to implement a `BaseChatClient` or `SupportsAgentRun` adapter instead of forking an unrelated provider. Start with the current From 24752f51dbcd40d714145adbb42d09c8e9a3ef7b Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Wed, 14 Oct 2026 20:22:57 +0200 Subject: [PATCH 07/10] Link volatile guidance to authoritative docs --- .../agent-framework-builder-python/SKILL.md | 9 +++++++++ .../references/hosting-and-deployment.md | 19 +++++-------------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/skills/agent-framework-builder-python/SKILL.md b/skills/agent-framework-builder-python/SKILL.md index f5c25591fd3..9b0da15a6b8 100644 --- a/skills/agent-framework-builder-python/SKILL.md +++ b/skills/agent-framework-builder-python/SKILL.md @@ -71,6 +71,15 @@ relying on remembered preview-era syntax. Use local guidance for stable decisions and common patterns. For provider matrices, package maturity, hosted tools, deployment details, or unfamiliar APIs, consult current sources before generating code: +When the coding agent supports MCP, configure the Microsoft Learn MCP Server as a Streamable HTTP server: + +```text +https://learn.microsoft.com/api/mcp +``` + +See [Microsoft Learn MCP Server setup](https://learn.microsoft.com/training/support/mcp-get-started) for client +configuration. The endpoint is for MCP clients and returns an error when opened directly in a browser. + 1. Search Microsoft Learn with `microsoft_docs_search`. 2. Fetch the relevant page with `microsoft_docs_fetch`. 3. Use `microsoft_code_sample_search` when a current implementation example is needed. diff --git a/skills/agent-framework-builder-python/references/hosting-and-deployment.md b/skills/agent-framework-builder-python/references/hosting-and-deployment.md index 725f7cd8699..7da06e8af1f 100644 --- a/skills/agent-framework-builder-python/references/hosting-and-deployment.md +++ b/skills/agent-framework-builder-python/references/hosting-and-deployment.md @@ -139,17 +139,9 @@ the application did not intend to use. ## Deploy a hosted agent to Microsoft Foundry -After local readiness is proven, follow the Foundry Hosted Agent deployment flow rather than inventing manifests or -infrastructure files: - -1. Start from a compatible `agent.manifest.yaml` for the selected Responses or Invocations host. -2. Initialize the deployment project with `azd ai agent init -m `. -3. Run `azd provision` only when the required Foundry project, model deployment, registry, and monitoring resources do - not already exist. -4. Test the host locally with `azd ai agent run` and `azd ai agent invoke --local`. -5. Deploy with `azd deploy`. - -Use these instructions according to the scenario: +Foundry deployment commands, manifests, prerequisites, injected environment variables, and supported deployment paths +change independently from Agent Framework. Do not reproduce or infer them from memory. After local readiness is proven, +open the current authoritative guide for the scenario: - [Agent Framework: Foundry hosted agents and deployment](https://learn.microsoft.com/agent-framework/hosting/foundry-hosted-agent#deploying-to-foundry) - [Foundry quickstart: deploy your first hosted agent with azd](https://learn.microsoft.com/azure/foundry/agents/quickstarts/quickstart-hosted-agent) @@ -159,9 +151,8 @@ Use these instructions according to the scenario: - [Manage deployed hosted agents](https://learn.microsoft.com/azure/foundry/agents/how-to/manage-hosted-agent) - [Agent Framework hosted-agent sample deployment instructions](https://github.com/microsoft/agent-framework/tree/main/python/samples/04-hosting/foundry-hosted-agents#deploying-the-agent-to-foundry) -Use dedicated Foundry tooling for provisioning, deployment, invocation, traces, RBAC, quota, capacity, and -troubleshooting. Host manifests and runtime environment variables are deployment contracts; copy them from the current -guide or sample for the selected protocol instead of constructing them from memory. +Use the selected guide or sample as the source of truth for `azd`, provisioning, deployment, invocation, traces, RBAC, +quota, capacity, troubleshooting, manifests, and runtime configuration. Current examples: From d537ec360560f2146d7812affcfeb10f53eb0d87 Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Thu, 15 Oct 2026 12:22:35 +0200 Subject: [PATCH 08/10] Align consumer skill detail with documentation --- .../agent-framework-builder-python/SKILL.md | 5 +- .../references/agent-builder.md | 71 +------ .../references/declarative.md | 103 +++------ .../references/evaluation.md | 98 ++------- .../references/harness-agent-builder.md | 196 ++++-------------- .../references/hosting-and-deployment.md | 180 ++++------------ .../references/integrations.md | 138 +++--------- .../references/observability.md | 60 ++---- .../references/workflow-builder.md | 154 +++----------- 9 files changed, 209 insertions(+), 796 deletions(-) diff --git a/skills/agent-framework-builder-python/SKILL.md b/skills/agent-framework-builder-python/SKILL.md index 9b0da15a6b8..ece1a705188 100644 --- a/skills/agent-framework-builder-python/SKILL.md +++ b/skills/agent-framework-builder-python/SKILL.md @@ -2,7 +2,6 @@ name: agent-framework-builder-python description: Use this skill when building, modifying, reviewing, evaluating, integrating, or hosting Python applications with Microsoft Agent Framework. Use it for Agent Framework agents, harness agents, workflows, orchestrations, declarative definitions, tools, sessions, middleware, evaluation, observability, integrations, or hosting, even when the user has not yet chosen the correct framework pattern. license: MIT -compatibility: Agent Skills-compatible coding agents; examples use Python 3.13 and current Microsoft Agent Framework packages. metadata: version: "0.3" project: agent-framework @@ -71,6 +70,10 @@ relying on remembered preview-era syntax. Use local guidance for stable decisions and common patterns. For provider matrices, package maturity, hosted tools, deployment details, or unfamiliar APIs, consult current sources before generating code: +The reference pages intentionally describe selection rules, invariants, and non-obvious gotchas. Use public docstrings +or API reference pages for signatures and option details, and use Learn or maintained samples for end-to-end +procedures. + When the coding agent supports MCP, configure the Microsoft Learn MCP Server as a Streamable HTTP server: ```text diff --git a/skills/agent-framework-builder-python/references/agent-builder.md b/skills/agent-framework-builder-python/references/agent-builder.md index e4dab1d9f46..a1bbbdac914 100644 --- a/skills/agent-framework-builder-python/references/agent-builder.md +++ b/skills/agent-framework-builder-python/references/agent-builder.md @@ -17,62 +17,17 @@ Capture: Ask only about unknowns that materially affect the design. -## Minimal project +## Start from maintained examples -```text -my-agent/ - pyproject.toml - agent.py -``` +For a new application, select the provider first and install only its required distribution and capabilities. Use the +current quickstart or maintained sample for package commands and runnable code rather than copying versions or setup +steps into the skill. -For the canonical Foundry-backed starting point, use the package manager selected by the project or user: +- [Your first agent](https://learn.microsoft.com/agent-framework/get-started/your-first-agent) +- [Python getting-started samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/01-get-started) -```bash -# uv -uv init --bare --python 3.13 -uv add agent-framework-foundry azure-identity - -# pip -python -m pip install agent-framework-foundry azure-identity -``` - -Install only the distributions required by the selected provider and capabilities. Add prerelease resolution only when -one of those distributions is prerelease. - -## Minimal Foundry agent - -Use environment variables in real applications: - -```python -from __future__ import annotations - -import asyncio -import os - -from agent_framework import Agent -from agent_framework.foundry import FoundryChatClient -from azure.identity import AzureCliCredential - - -async def main() -> None: - agent = Agent( - client=FoundryChatClient( - project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], - model=os.environ["FOUNDRY_MODEL"], - credential=AzureCliCredential(), - ), - name="my-agent", - instructions="Describe the agent behavior here.", - ) - print(await agent.run("Hello")) - - -if __name__ == "__main__": - asyncio.run(main()) -``` - -Agent Framework does not automatically load `.env` files. Add `python-dotenv` and call `load_dotenv()` only when the -application intentionally uses a `.env` file. +The stable pattern is an `Agent` wrapping a chat client, instructions, tools, middleware, and ContextProviders. Agent +Framework does not automatically load `.env` files; configuration loading remains application-owned. ## Provider selection @@ -104,16 +59,12 @@ JSON schemas. ## Sessions -Use an `AgentSession` for multi-turn state: - -```python -session = agent.create_session() -await agent.run("Remember Amsterdam.", session=session) -response = await agent.run("What city should you remember?", session=session) -``` +Create an `AgentSession` through the agent and pass the same session to each run in one logical conversation. - Do not reuse a session concurrently unless the provider and application explicitly support it. - Treat persisted session state as sensitive data. +- Read [session documentation](https://learn.microsoft.com/agent-framework/concepts/agents/conversations/session) for + serialization and provider-specific continuation behavior. ## History diff --git a/skills/agent-framework-builder-python/references/declarative.md b/skills/agent-framework-builder-python/references/declarative.md index 240ce20d752..b2adb87cb02 100644 --- a/skills/agent-framework-builder-python/references/declarative.md +++ b/skills/agent-framework-builder-python/references/declarative.md @@ -1,89 +1,36 @@ # Declarative agents and workflows -Use declarative construction when YAML or JSON should be the source of truth for a supported agent or workflow -definition. Keep credentials, custom tools, transports, authorization, and application services in code. +Use declarative construction when YAML or JSON should be the source of truth for supported agent or workflow behavior. +Keep credentials, custom tools, transports, authorization, and application services in code. -## Package and maturity +## Selection rules -`agent-framework-declarative` is a released distribution: +- Use `AgentFactory` for declarative agents and `WorkflowFactory` for declarative workflows. +- Prefer code-first construction when custom Python logic, strong typing, unsupported actions, or dynamic routing would + make the definition harder to validate than code. +- Do not invent schema kinds or parse definitions into ad hoc constructor calls; use the published schema and factory. -```bash -uv add agent-framework-declarative -``` +The `agent-framework-declarative` distribution is released, but individual surfaces can have different feature +maturity. Check current package status and public docstrings before making compatibility commitments. -Add the selected provider distribution separately. The package being released does not make every API stable: +## Host responsibilities -- Declarative agent loading through `AgentFactory` is experimental. -- Declarative workflow construction through `WorkflowFactory` is not feature-stage decorated. +- Supply provider clients, connections, bindings, tools, and external handlers explicitly. +- Provide an HTTP or MCP handler when a workflow uses those actions. +- Keep environment exposure restricted to intentional configuration. +- Treat definitions as executable configuration and load them only from trusted sources. +- Allowlist external destinations and register only resources the definition may invoke. +- Review the documented HTTP body limits and handler ownership rules; use a custom handler when host policy requires + different authentication, retries, SSRF controls, or transport behavior. -Check current package and feature maturity before making compatibility commitments. +## Validation -## Declarative agents +Validate definitions at startup and fail clearly on unknown providers, actions, tools, variables, or resources. Apply +the same approval, evaluation, and telemetry policy used for code-first agents and workflows. -Use `AgentFactory` to create an agent from inline YAML or a file: +## Learn more -```python -import os - -from agent_framework.declarative import AgentFactory -from agent_framework.foundry import FoundryChatClient -from azure.identity import AzureCliCredential - - -client = FoundryChatClient( - project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], - model=os.environ["FOUNDRY_MODEL"], - credential=AzureCliCredential(), -) -factory = AgentFactory(client=client) -agent = factory.create_agent_from_yaml_path("agent.yaml") -``` - -The application remains responsible for provider configuration, credentials, allowed bindings, connections, and -tools. `AgentFactory` emits the experimental feature warning. - -See [declarative agents](https://learn.microsoft.com/agent-framework/agents/declarative). - -## Declarative workflows - -Use `WorkflowFactory` for supported YAML workflow actions: - -```python -from agent_framework.declarative import WorkflowFactory - - -factory = WorkflowFactory() -workflow = factory.create_workflow_from_yaml_path("workflow.yaml") -``` - -Supply application-owned agents and bindings explicitly. Provide an HTTP or MCP handler when the workflow uses those -actions; build should fail rather than silently dispatch through an unconfigured transport. - -See [declarative workflows](https://learn.microsoft.com/agent-framework/workflows/declarative). - -## When to stay code-first - -Prefer `Agent`, a functional workflow, `WorkflowBuilder`, or an orchestration builder when: - -- custom Python logic is the primary behavior; -- direct typing and refactoring matter more than configuration portability; -- the required executor, action, provider, tool, or middleware is not represented by the declarative schema; -- dynamic routing would make the YAML harder to validate than code. - -Do not invent unsupported YAML kinds. Use current schema and examples. - -## Security and validation - -- Treat declarative definitions as executable application configuration. -- Load definitions only from trusted or authenticated sources. -- Keep secrets and credentials outside YAML. -- Keep environment exposure restricted to explicitly supplied configuration by default. -- Allowlist HTTP and MCP destinations. -- Register only the tools, agents, and services the definition may invoke. -- Use custom handlers for authentication, retries, SSRF controls, and connection ownership. -- `DefaultHttpRequestHandler` limits wire and decoded response bodies to 10 MiB by default, including intermediate - redirect and authentication responses. Set `max_response_body_bytes` when the host requires a different positive - bound, and do not use response hooks or transports that buffer the body before the handler enforces it. -- Validate at startup and fail clearly on unknown providers, actions, tools, variables, or resources. -- Test declarative agents and workflows with the same approval, evaluation, and telemetry policies as code-first - construction. +- [Declarative agents](https://learn.microsoft.com/agent-framework/agents/declarative) +- [Declarative workflows](https://learn.microsoft.com/agent-framework/workflows/declarative) +- [Declarative package README](https://github.com/microsoft/agent-framework/blob/main/python/packages/declarative/README.md) +- [Declarative samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/02-agents/declarative) diff --git a/skills/agent-framework-builder-python/references/evaluation.md b/skills/agent-framework-builder-python/references/evaluation.md index f528dea7d11..3b50930074c 100644 --- a/skills/agent-framework-builder-python/references/evaluation.md +++ b/skills/agent-framework-builder-python/references/evaluation.md @@ -1,92 +1,40 @@ # Evaluation -Use evaluation to turn expected behavior, failures, and user feedback into repeatable checks. +Use evaluation to turn expected behavior, failures, and feedback into repeatable evidence. Evaluation determines whether +behavior is acceptable; observability explains what happened during execution. -The local evaluation APIs and Foundry evaluation APIs are currently experimental. Keep their use isolated and review -feature maturity before exposing them as a long-term application contract. +Evaluation APIs are experimental. Check current feature maturity and API docstrings before making them a long-term +application contract. -## Capture a useful regression case +## Capture the case -Record: - -- input prompt or triggering event; -- current and expected output; -- expected tool calls and important arguments; -- relevant conversation, session, or workflow state; -- provider and model configuration; -- agent or workflow version; -- user feedback and the behavior it implies. - -Translate feedback into an expected output, expected tool call, context fixture, or custom evaluator instead of -inventing an unsupported framework field. +Record the input, current and expected output, expected tool calls, relevant session or workflow state, provider/model, +application version, and actionable user feedback. Express feedback as expected behavior or a custom evaluator instead +of inventing unsupported fields. ## Diagnose before changing -Classify the failure: +Classify the failure before modifying prompts or models: -1. deterministic code, schema, or tool bug; -2. wrong tool selection or approval behavior; +1. deterministic code, schema, or tool behavior; +2. tool selection or approval; 3. missing context, retrieval, memory, or history; -4. ambiguous instructions or output contract; +4. instructions or output contract; 5. workflow routing, state, checkpoint, or output selection; -6. provider or client configuration; -7. hosting or runtime environment; -8. model limitation after application-layer causes are excluded. - -Change the lowest layer that caused the failure. - -## Local evaluation - -```python -from agent_framework import LocalEvaluator, evaluate_agent, keyword_check, tool_called_check - - -evaluator = LocalEvaluator( - keyword_check("weather"), - tool_called_check("get_weather"), -) - -results = await evaluate_agent( - agent=agent, - queries=["What is the weather in Amsterdam?"], - expected_output=["A concise weather answer."], - evaluators=evaluator, -) -``` - -Useful APIs include: - -- `EvalItem`, `EvalResults`, `Evaluator`, and `LocalEvaluator`; -- `@evaluator` for custom checks; -- `ExpectedToolCall`; -- `evaluate_agent(...)` and `evaluate_workflow(...)`; -- `keyword_check`, `tool_called_check`, `tool_calls_present`, and tool-argument checks. - -Use exact string comparison only when exact wording is part of the contract. Prefer deterministic checks for tools, -schemas, required facts, and output structure before adding model-based graders. - -See [agent evaluation](https://learn.microsoft.com/agent-framework/agents/evaluation). - -## Foundry evaluation - -Use `FoundryEvals`, `evaluate_foundry_target`, or `evaluate_traces` when cloud evaluators, dashboards, or trace-derived -evaluation are required. These APIs are experimental. +6. provider, hosting, or environment configuration; +7. model limitation after application causes are excluded. -Keep local deterministic checks for fast inner-loop and CI coverage. Treat prompt optimization and continuous -monitoring as platform workflows rather than implicit Agent Framework behavior. +Fix the lowest responsible layer. Prefer deterministic checks for tools, schemas, required facts, and output structure; +use model-based or Foundry evaluators only where they add signal. ## Improvement loop -1. Reproduce the failure. -2. Add or update a regression case. -3. Fix the responsible layer. -4. Run deterministic checks. -5. Run model-based or Foundry evaluators only where they add signal. -6. Compare results. -7. Keep the case after the fix. +Reproduce the failure, add a regression case, fix the responsible layer, run deterministic checks, add model-based +evaluation if needed, compare results, and retain the case. -Current examples: +## Learn more -- [Local agent evaluation](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/evaluation/evaluate_agent.py) -- [Expected outputs and tool calls](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/evaluation/evaluate_with_expected.py) -- [Workflow evaluation](https://github.com/microsoft/agent-framework/blob/main/python/samples/03-workflows/evaluation/evaluate_workflow.py) +- [Agent evaluation](https://learn.microsoft.com/agent-framework/agents/evaluation) +- [Local evaluation samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/02-agents/evaluation) +- [Workflow evaluation sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/03-workflows/evaluation/evaluate_workflow.py) +- [Foundry evaluation samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/05-end-to-end/evaluation) diff --git a/skills/agent-framework-builder-python/references/harness-agent-builder.md b/skills/agent-framework-builder-python/references/harness-agent-builder.md index e8c567b1976..ad992ce0a44 100644 --- a/skills/agent-framework-builder-python/references/harness-agent-builder.md +++ b/skills/agent-framework-builder-python/references/harness-agent-builder.md @@ -1,175 +1,51 @@ # Harness agent builder -Use this path when the application needs a batteries-included agent operating loop rather than an explicit workflow -graph. +Use a harness when the product needs an opinionated operating loop around a normal `Agent`: planning, todos, memory, +approval handling, compaction, or optional autonomous capabilities. -`create_harness_agent` is released. Some capabilities it can enable are still experimental or prerelease, so assess -them independently instead of labeling the entire harness experimental. +`create_harness_agent` is a convenience composition, not a separate agent abstraction. It returns an `Agent` +implementing `SupportsAgentRun`. Everything it assembles can also be composed manually from an `Agent`, middleware, +context providers, history, and instructions. -The factory is a convenience composition over the regular `Agent` building blocks, not a separate runtime. Both -directions are valid: +## Choose the composition direction -- Start with `create_harness_agent` to get the full opinionated feature set, then disable or replace capabilities the - application does not need. -- Start with `Agent` and add the same instructions, chat middleware, agent middleware, context providers, history, - compaction, approval, and loop behavior explicitly. +- Start with `create_harness_agent` and disable or replace features when the product wants most harness capabilities. +- Start with `Agent` and add individual components when the product needs only a few capabilities or exact ordering. +- Use a workflow instead when explicit graph topology, routing, checkpointed stages, or intermediate outputs define + correctness. -Both approaches produce a normal agent implementing `SupportsAgentRun`; choose based on whether fast assembly or -fine-grained composition is more important. +## Important defaults and opt-ins -## When to choose a harness +Keep only the defaults that affect architecture or safety in working memory; read the +[harness documentation](https://learn.microsoft.com/agent-framework/concepts/harness) or the factory docstring for the +complete option list. -Choose `create_harness_agent` when the agent should: +- Todo, plan/execute mode, file memory, tool approval, history persistence, and telemetry are enabled by default. +- Compaction is active only when token budgets or custom strategies are supplied. +- Web search is added only when the client supports it and it is not disabled. +- File access, Agent Skills, background agents, shell execution, and looping are opt-in. +- Background agents, file access, looping, and shell tooling have experimental or prerelease constraints; check + current feature maturity before depending on them. -- plan work with todos and plan/execute modes; -- keep file-backed, session-scoped memory; -- apply tool approval and standing approval rules; -- optionally load Agent Skills or use provider-supported web search; -- optionally access files, delegate to background agents, execute shell commands, or loop until work completes; -- expose the normal `Agent` interface to callers. +## Operational gotchas -Use a workflow when graph topology, routing, checkpointing, or caller-visible intermediate outputs define correctness. - -## Minimal setup - -```python -from __future__ import annotations - -import asyncio -import os - -from agent_framework import create_harness_agent -from agent_framework.foundry import FoundryChatClient -from azure.identity import AzureCliCredential - - -async def main() -> None: - agent = create_harness_agent( - client=FoundryChatClient( - project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], - model=os.environ["FOUNDRY_MODEL"], - credential=AzureCliCredential(), - ), - name="my-harness-agent", - agent_instructions="Help users plan and complete multi-step tasks.", - disable_file_memory=True, - disable_web_search=True, - ) - session = agent.create_session() - print(await agent.run("Plan a weekend trip.", session=session)) - - -if __name__ == "__main__": - asyncio.run(main()) -``` - -The session is important when the default tool auto-approval middleware is enabled because standing approval state is -session-scoped. - -## Capability defaults - -| Capability | Current behavior | -| --- | --- | -| Function invocation and per-service-call history persistence | Enabled by the harness | -| Todo provider and plan/execute mode | Enabled unless disabled or replaced | -| File memory | Enabled by default; backed by a filesystem store unless replaced | -| File access | Opt-in through `file_access_store`; experimental | -| Tool auto-approval middleware | Enabled unless disabled; still respects tool approval requirements | -| Web search | Added only when the client supports it and it is not disabled | -| Compaction | Enabled when token budgets or custom compaction strategies are supplied | -| Skills | Opt-in through `skills_paths` or `skills_provider` | -| Background agents | Opt-in and experimental | -| Shell | Opt-in, prerelease, and requires `agent-framework-tools` | -| Looping | Opt-in and experimental; bound with `loop_max_iterations` | -| OpenTelemetry | Wired through the harness telemetry layer | - -Do not describe an optional capability as active unless its required option is present. - -## Compaction - -Supply both budgets for the default token-aware strategies: - -```python -agent = create_harness_agent( - client=client, - max_context_window_tokens=128_000, - max_output_tokens=16_384, - agent_instructions="Research carefully and cite reliable sources.", -) -``` - -Without token budgets or custom before/after strategies, compaction is disabled. - -## File memory and file access - -- File memory is session-scoped and enabled by default. Treat it as persisted application data. -- File access is separate and opt-in through an `AgentFileStore`, commonly `FileSystemAgentFileStore`. -- File access tools require approval by default. -- Prefer `FileAccessProvider.read_only_tools_auto_approval_rule` when reads may run automatically but writes must - remain approval-gated. -- Use `file_access_disable_write_tools=True` when the application must be read-only. -- Keep stores rooted in an intentional application directory. -- For hosted or untrusted workloads, do not use the shared default `{cwd}/agent-file-memory` store without operational - controls. Either disable file memory or supply tenant-scoped storage with enforced byte and file-count quotas, - retention limits, and server-owned cleanup. File-memory writes do not require tool approval, so storage isolation - and resource limits must be enforced by the host. - -Example source: -[Harness data processing](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/harness/harness_data_processing.py). - -## Skills - -Use `skills_paths` for file-based Agent Skills and `skills_provider` for custom or code-defined skills. When neither is -provided, no `SkillsProvider` is added. - -- Load skills only from trusted locations. -- Treat externally supplied skill content as untrusted input. -- Keep skill loading, resource reads, and script execution approval-gated unless the application intentionally changes - those policies. - -## Looping and background work - -Looping is opt-in: - -```python -from agent_framework import create_harness_agent, todos_remaining, todos_remaining_message - -agent = create_harness_agent( - client=client, - loop_should_continue=todos_remaining(looping_modes=["execute"]), - loop_next_message=todos_remaining_message, - loop_max_iterations=10, -) -``` - -- Always set a reasonable iteration cap. -- Stop for pending approvals or user input. -- Give background agents unique names, narrow responsibilities, and explicit result contracts. -- Treat background-agent input and output as potential prompt-injection and data-exfiltration boundaries. - -Looping and background agents currently emit experimental warnings. - -## Shell integration - -Pass `shell_executor` only when command execution is an explicit product requirement. - -- Install the prerelease `agent-framework-tools` distribution for the supplied shell implementations. -- Confine the working directory and environment. -- Keep command execution approval-gated. -- Treat command filters as a usability safeguard, not a security boundary. -- The caller owns the executor lifecycle. +- Use an `AgentSession` when standing approval state or multi-turn behavior matters. +- File memory and file access are separate. File access tools require approval by default; file-memory writes do not. +- For hosted or untrusted workloads, disable file memory or supply tenant-scoped storage with quotas, retention, and + server-owned cleanup. Do not rely on the shared default working-directory store. +- Load skills and background agents only from trusted sources; their content crosses the model and tool boundary. +- Bound autonomous loops and stop for unresolved approvals or user input. +- Treat shell command filters as a usability guard, not a security boundary. The host owns sandboxing and executor + lifecycle. ## Validation -1. Construct the harness and create a session. -2. Run a trivial prompt. -3. Exercise todos and modes if enabled. -4. Verify file memory and file-access roots. -5. Verify approval behavior for a side-effecting tool. -6. Prove loop termination and iteration limits when looping is enabled. -7. Clean up shell and background-agent resources. +- Exercise enabled modes, memory, skills, and file roots rather than testing factory construction alone. +- Verify approval behavior for side effects. +- Prove loop termination and resource cleanup for optional autonomous features. -Current examples: +## Learn more -- [First harness](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/06_agent_harness.py) -- [Research harness](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/harness/harness_research.py) +- [Harness concepts](https://learn.microsoft.com/agent-framework/concepts/harness) +- [First harness sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/06_agent_harness.py) +- [Harness samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/02-agents/harness) diff --git a/skills/agent-framework-builder-python/references/hosting-and-deployment.md b/skills/agent-framework-builder-python/references/hosting-and-deployment.md index 7da06e8af1f..eac727bd81c 100644 --- a/skills/agent-framework-builder-python/references/hosting-and-deployment.md +++ b/skills/agent-framework-builder-python/references/hosting-and-deployment.md @@ -1,161 +1,51 @@ # Hosting and deployment -Use this path after the agent or workflow works locally. Keep Agent Framework application readiness separate from -cloud provisioning, RBAC, quota, networking, and deployment operations. - -Hosting packages have independent lifecycle states. Verify current package maturity and host contracts before -generating files. +Use hosting after the agent or workflow works locally. Separate the runtime protocol from the deployment platform: +hosting libraries expose an application; infrastructure tooling deploys it. ## Choose the hosting family -| Need | Hosting family | -| --- | --- | -| Local execution | Direct Python process | -| Protocol hosting in any self-managed environment | General `agent-framework-hosting*` packages | -| Microsoft Foundry Agent Server | `agent-framework-foundry-hosting` | -| Durable execution or Azure Functions | Durable Agent Framework extension | - -Start with [hosting guidance](https://learn.microsoft.com/agent-framework/get-started/hosting) and then open the -selected host's detailed documentation and current sample. - -## General hosting - -The general hosting packages expose Agent Framework applications through protocols without targeting a specific cloud -or deployment environment. The application owns the process, network endpoint, scaling, identity, persistence, and -deployment platform. - -| Protocol or surface | Package | -| --- | --- | -| Shared hosting abstractions | `agent-framework-hosting` | -| OpenAI-compatible Responses | `agent-framework-hosting-responses` | -| Agent-to-Agent (A2A) | `agent-framework-hosting-a2a` | -| Model Context Protocol (MCP) | `agent-framework-hosting-mcp` | -| Telegram | `agent-framework-hosting-telegram` | - -These packages provide host adapters and protocol behavior; they do not provision or deploy the surrounding -infrastructure. They are currently alpha packages, so verify their current contracts before depending on them. - -Treat protocol request bodies and translated run options as caller-controlled input. Forward only an explicit allowlist -of options that the application supports; do not pass caller-supplied tools, instructions, or -`additional_function_arguments` into `agent.run(...)` unless the application deliberately exposes those capabilities. - -## Foundry hosting - -`agent-framework-foundry-hosting` is specifically for the Microsoft Foundry Agent Server and currently provides at -least two hosting protocols: - -| Foundry protocol | API | +| Need | Source of detail | | --- | --- | -| OpenAI-compatible Responses | `ResponsesHostServer` | -| Custom invocation request and response contracts | `InvocationsHostServer` | - -This package integrates Agent Framework sessions, responses, approvals, workflows, and Foundry sandbox identity with -the platform host. It is not the general-purpose hosting layer for arbitrary environments. - -### Responses protocol - -`agent-framework-foundry-hosting` is currently beta, so allow prerelease resolution when installing it. - -```python -from __future__ import annotations - -import os - -from agent_framework import Agent -from agent_framework.foundry import FoundryChatClient, ResponsesHostServer -from azure.identity import DefaultAzureCredential - - -agent = Agent( - client=FoundryChatClient( - project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], - model=os.environ["FOUNDRY_MODEL"], - credential=DefaultAzureCredential(), - ), - instructions="You are a helpful assistant.", - default_options={"store": False}, -) - -ResponsesHostServer(agent=agent, history_source="agent_server").run() -``` - -Choose the history source deliberately: - -- `agent_server` lets the host reconstruct outer Responses history and disables duplicate inner storage; -- `service` uses the downstream service's private continuation; -- `agent` lets the agent's history provider or client storage own history. - -Do not combine competing history owners. - -### Invocations protocol - -Use `InvocationsHostServer` when the hosted application needs its own invocation payload and response contract instead -of the OpenAI Responses protocol. Supply the documented parser or invocation handler and make session, identity, and -workflow continuation behavior explicit. - -## Foundry agent and workflow ownership - -- Passing an agent instance reuses it for the host lifetime. -- Pass a zero-argument sync or async factory when the agent owns mutable state outside `AgentSession`. -- Prefer native `workflow=` hosting when the host supports workflows. -- Hosting `workflow.as_agent()` is deprecated in Foundry hosting because a shared workflow agent is stateful and can - leak state across requests. -- Workflow factories must build fresh workflows, executors, agents, clients, providers, and mutable tools while - preserving stable workflow and executor identities for checkpoint restoration. - -## Entry-point contracts - -- `build_agent()` is an application convention, not a framework requirement. -- Export a module-level `agent` only when the selected discovery or hosting path requires it. -- Foundry host servers accept an agent instance or factory through `agent=`. -- Native Foundry workflow hosting has its own `workflow=` and request-parsing contracts. -- General Responses, A2A, MCP, Telegram, and durable hosts use their own adapters; follow the selected package's - examples instead of applying Foundry-specific server contracts. - -## Configuration and identity - -Document only variables required by the selected provider and host. Common Foundry examples use: +| Self-managed protocol endpoint | [Agent Framework hosting](https://learn.microsoft.com/agent-framework/hosting/) | +| Microsoft Foundry Agent Server | [Foundry hosted agents](https://learn.microsoft.com/agent-framework/hosting/foundry-hosted-agent) | +| Durable execution or Azure Functions | [Durable Agent Framework extension](https://github.com/microsoft/agent-framework-durable-extension) | -- `FOUNDRY_PROJECT_ENDPOINT`; -- `FOUNDRY_MODEL`; -- a development credential such as `AzureCliCredential`; -- a deliberate production credential, commonly managed identity. +General hosting packages target arbitrary environments and provide protocol adapters such as Responses, A2A, MCP, or +Telegram. `agent-framework-foundry-hosting` is specific to the Foundry Agent Server and supports Responses and +Invocations contracts. Check current package maturity and protocol documentation rather than copying option inventories +into application code. -Do not commit secrets. Review `DefaultAzureCredential` for production because its fallback chain may probe credentials -the application did not intend to use. +## Stable hosting rules -## Deployment readiness checklist +- Treat translated request options as caller-controlled input and forward only an explicit allowlist. +- Choose one history owner; do not combine competing host, service, and agent history mechanisms. +- Reuse an agent instance only when all mutable state is session-scoped. Otherwise use a request-owned factory. +- Prefer native workflow hosting. A shared `workflow.as_agent()` instance can leak workflow state across requests. +- Build fresh workflows and request-owned resources while keeping workflow and executor identities stable for + checkpoint restoration. +- Make session, checkpoint, response, identity, approval, storage, and cleanup ownership explicit. -- `pyproject.toml` declares only required runtime packages. -- The selected entry point imports and constructs the agent or workflow. -- Required environment variables fail with clear messages. -- Session, checkpoint, response, and service-history ownership are explicit. -- Tool side effects and approval behavior are documented. -- External files, skills, prompts, and resources are packaged intentionally. -- Health, logging, telemetry, and sensitive-data settings match the host. -- Request-scoped resources are created and cleaned up by the request-scoped factory. -- The deployment package contains no local credentials, caches, or unrelated files. -- Host-specific manifests are added only by the deployment workflow that consumes them. +Read the selected host's public docstrings and maintained sample for constructor options, parsers, history modes, +storage providers, and protocol details. -## Deploy a hosted agent to Microsoft Foundry +## Deployment readiness -Foundry deployment commands, manifests, prerequisites, injected environment variables, and supported deployment paths -change independently from Agent Framework. Do not reproduce or infer them from memory. After local readiness is proven, -open the current authoritative guide for the scenario: +Verify the entry point imports, required packages and external files are included, configuration fails clearly, history +and storage ownership are documented, request-owned resources are cleaned up, and no local credentials or caches enter +the deployment package. -- [Agent Framework: Foundry hosted agents and deployment](https://learn.microsoft.com/agent-framework/hosting/foundry-hosted-agent#deploying-to-foundry) -- [Foundry quickstart: deploy your first hosted agent with azd](https://learn.microsoft.com/azure/foundry/agents/quickstarts/quickstart-hosted-agent) -- [Foundry: deploy a hosted agent](https://learn.microsoft.com/azure/foundry/agents/how-to/deploy-hosted-agent) -- [Foundry: deploy a hosted agent from source code](https://learn.microsoft.com/azure/foundry/agents/how-to/deploy-hosted-agent-code) -- [Host Agent Framework agents as Foundry hosted agents](https://learn.microsoft.com/azure/foundry/how-to/develop/framework-hosted-agents#deploy) -- [Manage deployed hosted agents](https://learn.microsoft.com/azure/foundry/agents/how-to/manage-hosted-agent) -- [Agent Framework hosted-agent sample deployment instructions](https://github.com/microsoft/agent-framework/tree/main/python/samples/04-hosting/foundry-hosted-agents#deploying-the-agent-to-foundry) +## Deploy to Microsoft Foundry -Use the selected guide or sample as the source of truth for `azd`, provisioning, deployment, invocation, traces, RBAC, -quota, capacity, troubleshooting, manifests, and runtime configuration. +Foundry deployment commands, manifests, prerequisites, and injected configuration change independently from Agent +Framework. Use current authoritative instructions: -Current examples: +- [Agent Framework Foundry deployment guidance](https://learn.microsoft.com/agent-framework/hosting/foundry-hosted-agent#deploying-to-foundry) +- [Foundry hosted-agent quickstart](https://learn.microsoft.com/azure/foundry/agents/quickstarts/quickstart-hosted-agent) +- [Deploy a hosted agent](https://learn.microsoft.com/azure/foundry/agents/how-to/deploy-hosted-agent) +- [Deploy from source code](https://learn.microsoft.com/azure/foundry/agents/how-to/deploy-hosted-agent-code) +- [Manage hosted agents](https://learn.microsoft.com/azure/foundry/agents/how-to/manage-hosted-agent) +- [Agent Framework hosted-agent samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/04-hosting/foundry-hosted-agents) -- [First hosted agent](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/07_hosting.py) -- [Foundry hosting package guidance](https://github.com/microsoft/agent-framework/blob/main/python/packages/foundry_hosting/README.md) -- [Foundry hosted-agent samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/04-hosting/foundry-hosted-agents) +Use those sources for `azd`, provisioning, manifests, environment variables, deployment, invocation, RBAC, quota, +monitoring, and troubleshooting. Do not reconstruct deployment files from memory. diff --git a/skills/agent-framework-builder-python/references/integrations.md b/skills/agent-framework-builder-python/references/integrations.md index af3507ab23e..2b298e0f103 100644 --- a/skills/agent-framework-builder-python/references/integrations.md +++ b/skills/agent-framework-builder-python/references/integrations.md @@ -1,133 +1,47 @@ # Providers and integrations -Choose integrations only after the application contract is clear. A model provider or agent service supplies the -underlying intelligence; tools, memory, protocols, UI, and hosting add separate capabilities. +Choose integrations only after the application contract is clear. Provider and connector capabilities, package +maturity, and Python compatibility change more frequently than core Agent Framework patterns. -Package and provider availability changes more frequently than core agent patterns. Check current Microsoft Learn -documentation and package metadata before adding dependencies or promising feature parity. +## Version and package selection -## Python version compatibility +- Check each selected distribution's `Requires-Python` metadata and the current + [Python package status](https://github.com/microsoft/agent-framework/blob/main/python/PACKAGE_STATUS.md). +- Read the [Python README compatibility notes](https://github.com/microsoft/agent-framework/blob/main/python/README.md) + for current boundary-version limitations rather than copying that matrix into application guidance. +- Install only the selected integrations and allow prerelease packages only when required. -Agent Framework supports Python 3.11 through 3.15. Build new environments with the latest supported Python release, or -at least the previous release (N-1), unless a required integration has a narrower compatibility range. - -Python 3.14 and 3.15 support the core framework, with these current integration limitations: - -| Area | Affected versions | Limitation | -| --- | --- | --- | -| Declarative Power Fx | Python 3.14-3.15 | `agent-framework-declarative` installs without its `powerfx` dependency. Expressions beginning with `=` are not evaluated. | -| Redis | Python 3.15 | `agent-framework-redis` is unavailable because RedisVL requires Python earlier than 3.15. | -| Hyperlight | Python 3.15 | `agent-framework-hyperlight` is unavailable until a compatible sandbox backend wheel is published. | -| AG-UI A2UI | Python 3.15 | Base `agent-framework-ag-ui` works, but its optional `a2ui` extra is unavailable because the toolkit requires Python earlier than 3.15. | - -Check each selected distribution's `Requires-Python` metadata instead of assuming every integration supports the full -framework range. - -## Providers and agent services - -Common released distributions include: - -- `agent-framework-foundry` for Microsoft Foundry chat clients, managed agents, memory, embeddings, and Foundry evals; -- `agent-framework-openai` for OpenAI and Azure OpenAI Responses or Chat Completions clients; -- `agent-framework-github-copilot` for GitHub Copilot-backed scenarios. - -Other providers and agent services have independent lifecycle states and Python requirements. Install only the selected -distribution and allow prerelease packages only when required. - -If no existing provider matches the backend, use the -[custom provider guidance](extensibility.md#custom-providers) to implement a `BaseChatClient` or -`SupportsAgentRun` adapter instead of forking an unrelated provider. +## Provider selection Start with the current [model provider catalog](https://learn.microsoft.com/agent-framework/integrations/by-component/model-providers/) and [agent service catalog](https://learn.microsoft.com/agent-framework/integrations/by-component/agent-services/). -Verify: - -- authentication and identity; -- Python and package version support; -- streaming and structured output; -- hosted tools; -- application-managed versus service-managed history; -- tool approval and function-loop behavior. - -## Tools - -Choose local versus hosted execution deliberately: - -- function tools for application-owned Python functions; -- local MCP for caller-owned MCP clients and sessions; -- hosted MCP, web search, file search, or code interpreter only when the selected provider supports them; -- Agent Skills through `SkillsProvider` when the built agent should discover reusable capabilities. - -Keep local function and MCP side effects under the application's approval and security policy. Hosted execution still -requires explicit identity, data-boundary, and provider-capability review. - -## Memory, retrieval, and vector stores - -Use normal sessions and context providers first. Add a connector when persistence, scale, search, or an external data -model requires it. - -Current integration categories include: - -- chat history and context providers; -- Foundry memory; -- Azure AI Search; -- Cosmos DB and Redis; -- vector-store connectors such as PostgreSQL, MongoDB, Qdrant, DuckDB, Oracle, SQL Server, and DocumentDB; -- graph and third-party memory providers. - -Some connector packages are alpha or beta even when core is released. Check lifecycle state and supported operations. - -Define tenant, user, retention, deletion, authorization, and encryption boundaries before persisting conversation, -memory, or vector data. - -## Protocol and UI integrations - -- A2A for interoperable agent-to-agent communication. -- MCP hosting when the application exposes tools through MCP. -- ChatKit and other application-specific UI integrations where appropriate. - -Protocol integration does not replace authorization. Validate identities, payloads, exposed tools, and cross-tenant -behavior at every boundary. - -### AG-UI - -Use `agent-framework-ag-ui` to connect Agent Framework agents and workflows to AG-UI-compatible frontends. - -- `add_agent_framework_fastapi_endpoint(...)` exposes a `SupportsAgentRun` implementation or `Workflow` through a - FastAPI endpoint. -- `AgentFrameworkAgent` and `AgentFrameworkWorkflow` provide the protocol adapters. -- `AGUIChatClient` connects an Agent Framework application to an existing AG-UI server. -- AG-UI supports streaming events, tool calls, shared state, interrupts, approval resume, and thread snapshots. -- Use a thread-scoped workflow factory when workflow instances hold mutable runtime state. -- AG-UI thread and run IDs are correlation identifiers, not authentication or tenant-isolation boundaries. -- The application owns authorization, durable snapshot storage, scaling, and frontend rendering policy. - -Current starting point: -[AG-UI single-agent sample](https://github.com/microsoft/agent-framework/tree/main/python/samples/05-end-to-end/ag_ui_single_agent). +Verify authentication, streaming, structured output, hosted tools, history ownership, approval, and function-loop +support instead of assuming provider parity. If no provider matches, use +[Extensibility](extensibility.md#custom-providers). -## Durable and cloud integrations +## Capability categories -Durable Task and Azure Functions integrations are maintained in the -[Durable Agent Framework extension](https://github.com/microsoft/agent-framework-durable-extension), not in the main -Python package tree. +- Use function tools for application-owned code and MCP for external tool services. +- Use sessions and ContextProviders first; add memory, retrieval, or vector-store connectors when persistence, search, + scale, or an external data model requires them. +- A2A connects agents across services; AG-UI connects agents or workflows to compatible frontends; hosting adapters + expose protocols. None of these replace application authorization. +- Durable Task and Azure Functions integrations are maintained in the + [Durable Agent Framework extension](https://github.com/microsoft/agent-framework-durable-extension). -Foundry hosting and other host adapters have separate packages and lifecycle states. Read -`hosting-and-deployment.md` before adding host or deployment files. +For AG-UI, remember that thread and run IDs are correlation identifiers, not authentication boundaries. The +application owns authorization, durable snapshots, scaling, and rendering policy. ## Selection checklist -1. Confirm the provider or service and authentication model. -2. Check package lifecycle and Python support. -3. Check required provider features rather than assuming parity. -4. Add only the selected integration packages. -5. Define data, network, identity, approval, and retention boundaries. -6. Build a focused integration test or opt-in live smoke test. -7. Document required environment variables and permissions. +Confirm the provider/service, authentication model, required capabilities, package maturity, Python range, data and +tenant boundaries, approval policy, and required permissions. Add a focused integration test or opt-in live smoke test. -Current catalogs: +## Learn more - [Agent Framework integrations](https://learn.microsoft.com/agent-framework/integrations/) - [Python package status](https://github.com/microsoft/agent-framework/blob/main/python/PACKAGE_STATUS.md) - [Python samples](https://github.com/microsoft/agent-framework/tree/main/python/samples) +- [AG-UI sample](https://github.com/microsoft/agent-framework/tree/main/python/samples/05-end-to-end/ag_ui_single_agent) diff --git a/skills/agent-framework-builder-python/references/observability.md b/skills/agent-framework-builder-python/references/observability.md index 167846223cf..b3f7b3e1abe 100644 --- a/skills/agent-framework-builder-python/references/observability.md +++ b/skills/agent-framework-builder-python/references/observability.md @@ -1,50 +1,24 @@ # Observability -Use observability to diagnose and monitor agent runs, model calls, tool calls, workflow execution, latency, errors, and -token use. Observability provides runtime evidence; it does not define whether an output is correct. +Use observability to diagnose agent runs, model calls, tools, workflows, latency, errors, and token use. It provides +runtime evidence; it does not determine whether output is correct. -## OpenTelemetry configuration +## Stable setup rules -Agent Framework instrumentation is enabled by default, but the application must configure OpenTelemetry providers and -exporters. Narrow Agent Framework installations include `opentelemetry-api` but not the SDK. Install -`opentelemetry-sdk` and the exporter package required by the application's telemetry backend before using the helper: +- Agent Framework instrumentation is enabled by default, but applications must configure OpenTelemetry providers and + exporters. +- Narrow installations include `opentelemetry-api`, not the SDK. Install `opentelemetry-sdk` and the exporter required + by the chosen backend. +- Configure providers once during startup, or use application-owned OpenTelemetry setup and enable Agent Framework + instrumentation against it. +- Keep sensitive payload capture disabled unless the application explicitly accepts the exposure. +- Built-in redaction does not replace sanitization in custom telemetry, exporters, middleware, or exception messages. -```bash -python -m pip install opentelemetry-sdk opentelemetry-exporter-otlp -``` +Inspect run boundaries, model calls, tool and MCP calls, workflow transitions, checkpoints, approvals, retries, usage, +and hosted-service continuations. Use stable agent, workflow, and service identifiers. -The helper configures providers from arguments and standard environment variables: +## Learn more -```python -from agent_framework.observability import configure_otel_providers - - -configure_otel_providers(enable_sensitive_data=False) -``` - -- Call `configure_otel_providers(...)` once during application startup. -- Keep sensitive prompts, tool arguments, tool results, and model responses disabled by default. -- Use `enable_instrumentation(...)` only to control instrumentation when providers are configured elsewhere. -- Use traces to correlate agent runs, model calls, tools, MCP calls, and workflow executors. -- Give agents, harnesses, workflows, and services stable identifiers. -- When sensitive data is disabled, built-in Agent Framework telemetry redacts exception details from spans, events, - workflow telemetry, and failure logs, and does not attach serialized tool arguments to invocation metrics. -- Apply equivalent redaction and cardinality controls to application-defined telemetry and custom exporters. - -## What to inspect - -- Agent and workflow run boundaries. -- Model requests, responses, latency, and token usage. -- Function and MCP tool calls, including failures and approval pauses. -- Workflow executor transitions, checkpoint activity, and suspended runs. -- Provider errors, retries, and hosted-service continuations. - -Avoid using telemetry as a storage mechanism for application state or sensitive conversation history. - -See [agent observability](https://learn.microsoft.com/agent-framework/agents/observability) and -[workflow observability](https://learn.microsoft.com/agent-framework/workflows/observability). - -Current examples: - -- [Agent observability](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/observability/agent_observability.py) -- [Workflow observability](https://github.com/microsoft/agent-framework/blob/main/python/samples/02-agents/observability/workflow_observability.py) +- [Agent observability](https://learn.microsoft.com/agent-framework/agents/observability) +- [Workflow observability](https://learn.microsoft.com/agent-framework/workflows/observability) +- [Observability samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/02-agents/observability) diff --git a/skills/agent-framework-builder-python/references/workflow-builder.md b/skills/agent-framework-builder-python/references/workflow-builder.md index f139745617e..9cf55bf0851 100644 --- a/skills/agent-framework-builder-python/references/workflow-builder.md +++ b/skills/agent-framework-builder-python/references/workflow-builder.md @@ -1,143 +1,53 @@ # Workflow and orchestration builder -Use this path when explicit control flow, multiple participants, resumable progress, or caller-visible intermediate work -matters. +Use a workflow when execution order, routing, parallelism, multiple participants, checkpoints, human input, or +intermediate outputs are part of the product contract. -## Selection +## Choose the workflow API -| Need | API and package | +| Need | API | | --- | --- | -| Simple Pythonic async composition | Experimental `@workflow` from `agent-framework-core` | -| Cached calls, step events, or step-level checkpoint replay | Experimental `@step` inside `@workflow` | -| Explicit typed topology, edges, executors, routing, and outputs | `WorkflowBuilder` from `agent-framework-core` | -| Standard multi-agent collaboration pattern | Builder from released `agent-framework-orchestrations` | -| YAML-defined actions and routing | `WorkflowFactory` from `agent-framework-declarative`; read `declarative.md` | +| Native Python control flow | Experimental `@workflow` | +| Cached or checkpoint-replayable function calls | Experimental `@step` | +| Explicit typed graph and routing | `WorkflowBuilder` | +| Standard multi-agent collaboration | `agent-framework-orchestrations` builders | +| YAML-defined actions and routing | `WorkflowFactory`; read `declarative.md` | -Prefer a plain agent or harness when an inspectable execution path is not part of the requirement. +Read [workflow concepts](https://learn.microsoft.com/agent-framework/concepts/workflows/) and the relevant API +docstrings for signatures and execution options. -## Functional workflow +## Stable design rules -`@workflow` and `@step` are experimental. The decorator creates a stateless definition; call `build()` to create a -stateful workflow for one logical caller or session. - -```python -from agent_framework import workflow - - -async def normalize(text: str) -> str: - return text.strip().upper() - - -@workflow -async def text_workflow(text: str) -> str: - return await normalize(text) - - -result = await text_workflow.build().run(" hello ") -print(result.get_outputs()[0]) -``` - -Normal async functions work inside a functional workflow. Add `@step` only where caching, checkpoint replay, event -emission, or step-level observability justify it. - -## Graph workflow - -Use executors and edges when explicit routing and typed data flow matter: - -```python -from agent_framework import WorkflowBuilder, WorkflowContext, executor -from typing_extensions import Never - - -@executor(id="normalize") -async def normalize(text: str, ctx: WorkflowContext[str]) -> None: - await ctx.send_message(text.strip().upper()) - - -@executor(id="finish") -async def finish(text: str, ctx: WorkflowContext[Never, str]) -> None: - await ctx.yield_output(text) - - -workflow = WorkflowBuilder(start_executor=normalize).add_edge(normalize, finish).build() -result = await workflow.run(" hello ") -``` - -Build a fresh workflow when executors or wrapped agents hold mutable run-specific state. - -## Agents in workflows - -Agents can be workflow executors: - -```python -workflow = WorkflowBuilder(start_executor=writer).add_edge(writer, reviewer).build() -result = await workflow.run("Draft a product announcement.") -``` - -Decide whether each participant needs an independent session, service-managed history, tools, middleware, or context. -Do not assume one agent's session is valid for another. - -## Standard orchestrations - -Install the released orchestration package when using these builders: - -```bash -uv add agent-framework-orchestrations -``` - -Import from `agent_framework.orchestrations`: - -- `SequentialBuilder`: staged output from one participant to the next. -- `ConcurrentBuilder`: fan-out to participants and aggregate. -- `HandoffBuilder`: transfer control among configured specialists. -- `GroupChatBuilder`: managed multi-participant conversation. -- `MagenticBuilder`: manager/planner-driven collaboration. - -Use the standard builder before implementing an equivalent custom graph. - -## Output selection - -Configure `output_from` for caller-facing workflow output and `intermediate_output_from` for progress or supporting -outputs. Without explicit selection, output-capable executors use the workflow's documented default behavior. - -When exposing a workflow through `workflow.as_agent()`, verify that its start executor accepts agent message input and -that callers actually require the `Agent` contract. Prefer native workflow hosting where the host supports it. +- A functional workflow definition must be built before running; each built workflow owns mutable execution state. +- Build fresh workflows when executors or wrapped agents hold run-specific state. +- Any `SupportsAgentRun` implementation can participate where an agent executor is accepted. +- Use standard sequential, concurrent, handoff, group-chat, or Magentic builders before recreating those patterns. +- Select caller-facing output separately from intermediate progress. +- Wrap a workflow as an agent only when callers require the agent contract. Prefer native workflow hosting when the + host supports it. ## Agent-to-workflow migration -Converting one agent into a workflow changes contracts, state, outputs, and often deployment behavior: - -1. Record the current input, response, tools, middleware, context, session, and hosting contract. -2. Choose a functional workflow, graph workflow, or standard orchestration. -3. Split responsibilities only where the split changes behavior or observability. -4. Define data passed between stages. -5. Select final and intermediate outputs. -6. Preserve session and history behavior intentionally. -7. Revisit tool approval across every participant and branch. -8. Keep an agent adapter only when the external contract requires it. +Before splitting an agent into stages, record its input, response, tools, middleware, context, session, and hosting +contract. Split responsibilities only where the topology changes correctness or observability. Define the messages +between stages, final output, intermediate output, session ownership, and approval behavior explicitly. ## Gotchas -- Functional workflow definitions must be built before running. -- A built workflow is stateful and cannot run concurrently. -- Handoff relationships must be configured explicitly. -- Magentic participants need stable, unique names. +- Workflow instances are stateful and cannot run concurrently. - Checkpoint resume requires stable workflow, executor, and participant identities. -- Human approval of a tool call and human input at a workflow transition are different mechanisms. +- Sessions are not automatically interchangeable between participants. +- Tool approval and workflow human input are different suspension mechanisms. - Workflow streams include framework events in addition to application output. ## Validation -1. Run the smallest graph with deterministic executors before adding model calls. -2. Assert final and intermediate outputs separately. -3. Exercise every route or branch. -4. Test suspend/resume when human input or checkpoints are used. -5. Smoke-test streaming event types when callers consume streams. -6. Verify participant sessions, approval behavior, and error propagation. -7. Inspect a visualization when topology is non-trivial. +Exercise every route with deterministic executors first. Test final and intermediate outputs separately, then test +streaming, suspension/resume, participant sessions, approvals, and error propagation where used. -Current examples: +## Learn more -- [Functional workflow](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/05a_functional_workflow_basics.py) -- [Graph workflow](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/05c_first_graph_workflow.py) -- [Agents in a workflow](https://github.com/microsoft/agent-framework/blob/main/python/samples/03-workflows/_start-here/step2_agents_in_a_workflow.py) +- [Workflow concepts](https://learn.microsoft.com/agent-framework/concepts/workflows/) +- [Functional workflow sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/05a_functional_workflow_basics.py) +- [Graph workflow sample](https://github.com/microsoft/agent-framework/blob/main/python/samples/01-get-started/05c_first_graph_workflow.py) +- [Workflow samples](https://github.com/microsoft/agent-framework/tree/main/python/samples/03-workflows) From 9ce4bf2c143b0e6824b4128763def3106a79300b Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Thu, 15 Oct 2026 14:10:02 +0200 Subject: [PATCH 09/10] Clarify provider settings resolution --- .../references/agent-builder.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/skills/agent-framework-builder-python/references/agent-builder.md b/skills/agent-framework-builder-python/references/agent-builder.md index a1bbbdac914..b90e83a99a5 100644 --- a/skills/agent-framework-builder-python/references/agent-builder.md +++ b/skills/agent-framework-builder-python/references/agent-builder.md @@ -105,7 +105,9 @@ For custom providers, ContextProviders, and middleware, read ## Validation 1. Import and construct the client and agent without invoking the model where practical. -2. Validate required environment variables with clear errors. +2. Check the selected client or provider documentation for its environment keys and tell the user to set them. Let the + framework classes resolve and validate those settings. If values come from a `.env` file, pass the documented + environment-file option or call `load_dotenv()` before construction; do not manually resolve each setting. 3. Unit-test deterministic tools, schemas, context providers, and middleware. 4. Add an opt-in live smoke test only when provider access is available. 5. Exercise a second turn when session behavior is required. From 43ae2e791df31d09f225bc31a3a1b7484b614778 Mon Sep 17 00:00:00 2001 From: eavanvalkenburg Date: Thu, 15 Oct 2026 15:23:14 +0200 Subject: [PATCH 10/10] Address Agent Skills review feedback --- .github/scripts/validate_agent_skill_links.py | 66 +++++++++++++++ .../tests/test_validate_agent_skill_links.py | 82 +++++++++++++++++++ .github/workflows/agent-skills.yml | 22 ++--- .../agent-framework-builder-python/SKILL.md | 19 +++-- .../references/workflow-builder.md | 3 +- 5 files changed, 167 insertions(+), 25 deletions(-) create mode 100644 .github/scripts/validate_agent_skill_links.py create mode 100644 .github/tests/test_validate_agent_skill_links.py diff --git a/.github/scripts/validate_agent_skill_links.py b/.github/scripts/validate_agent_skill_links.py new file mode 100644 index 00000000000..1c986146159 --- /dev/null +++ b/.github/scripts/validate_agent_skill_links.py @@ -0,0 +1,66 @@ +#!/usr/bin/env python3 +# Copyright (c) Microsoft. All rights reserved. +"""Validate local Markdown resources bundled with public Agent Skills.""" + +from __future__ import annotations + +import re +import stat +import sys +from pathlib import Path + +LINK_PATTERN = re.compile(r"\[[^\]]+\]\(([^)]+)\)") + + +def validate_agent_skill_links(skills_directory: Path) -> list[str]: + """Return validation errors for local Markdown links in public Agent Skills.""" + errors: list[str] = [] + skill_roots = sorted(path.parent for path in skills_directory.rglob("SKILL.md")) + + if not skill_roots: + return [f"No Agent Skills found under {skills_directory}."] + + for skill_root in skill_roots: + resolved_root = skill_root.resolve(strict=True) + for markdown_path in skill_root.rglob("*.md"): + for target in LINK_PATTERN.findall(markdown_path.read_text(encoding="utf-8")): + if "://" in target or target.startswith(("#", "mailto:")): + continue + + relative_target = target.split("#", 1)[0] + if not relative_target: + continue + + target_path = Path(relative_target) + if target_path.is_absolute(): + errors.append(f"{markdown_path}: absolute local link is not allowed: {target}") + continue + + candidate = markdown_path.parent / target_path + try: + resolved_candidate = candidate.resolve(strict=True) + except FileNotFoundError: + errors.append(f"{markdown_path}: missing local resource: {target}") + continue + + try: + resolved_candidate.relative_to(resolved_root) + except ValueError: + errors.append(f"{markdown_path}: local resource escapes skill root: {target}") + continue + + if not stat.S_ISREG(candidate.lstat().st_mode): + errors.append(f"{markdown_path}: local resource is not a regular file: {target}") + + return errors + + +def main() -> None: + skills_directory = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("skills") + errors = validate_agent_skill_links(skills_directory) + if errors: + raise SystemExit("\n".join(errors)) + + +if __name__ == "__main__": + main() diff --git a/.github/tests/test_validate_agent_skill_links.py b/.github/tests/test_validate_agent_skill_links.py new file mode 100644 index 00000000000..a309104b826 --- /dev/null +++ b/.github/tests/test_validate_agent_skill_links.py @@ -0,0 +1,82 @@ +# Copyright (c) Microsoft. All rights reserved. + +# ruff:file-ignore[implicit-namespace-package, undocumented-public-class, undocumented-public-method] + +from __future__ import annotations + +import sys +import tempfile +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parents[1] / "scripts")) + +from validate_agent_skill_links import validate_agent_skill_links + + +class ValidateAgentSkillLinksTests(unittest.TestCase): + def setUp(self) -> None: + self.temp_dir = tempfile.TemporaryDirectory() + self.root = Path(self.temp_dir.name) + self.skills = self.root / "skills" + self.skill = self.skills / "example-skill" + self.skill.mkdir(parents=True) + + def tearDown(self) -> None: + self.temp_dir.cleanup() + + def write_skill(self, body: str) -> None: + (self.skill / "SKILL.md").write_text(body, encoding="utf-8") + + def test_accepts_regular_file_inside_skill(self) -> None: + (self.skill / "references").mkdir() + (self.skill / "references" / "details.md").write_text("# Details\n", encoding="utf-8") + self.write_skill("[Details](references/details.md)\n") + + self.assertEqual([], validate_agent_skill_links(self.skills)) + + def test_rejects_parent_traversal(self) -> None: + (self.skills / "outside.md").write_text("# Outside\n", encoding="utf-8") + self.write_skill("[Outside](../outside.md)\n") + + errors = validate_agent_skill_links(self.skills) + + self.assertTrue(any("escapes skill root" in error for error in errors)) + + def test_rejects_absolute_path(self) -> None: + absolute_file = self.root / "outside.md" + absolute_file.write_text("# Outside\n", encoding="utf-8") + self.write_skill(f"[Outside]({absolute_file})\n") + + errors = validate_agent_skill_links(self.skills) + + self.assertTrue(any("absolute local link" in error for error in errors)) + + def test_rejects_directory_target(self) -> None: + (self.skill / "references").mkdir() + self.write_skill("[References](references)\n") + + errors = validate_agent_skill_links(self.skills) + + self.assertTrue(any("not a regular file" in error for error in errors)) + + def test_rejects_symlink_target(self) -> None: + target = self.skill / "details.md" + target.write_text("# Details\n", encoding="utf-8") + (self.skill / "linked.md").symlink_to(target) + self.write_skill("[Details](linked.md)\n") + + errors = validate_agent_skill_links(self.skills) + + self.assertTrue(any("not a regular file" in error for error in errors)) + + def test_rejects_missing_target(self) -> None: + self.write_skill("[Missing](references/missing.md)\n") + + errors = validate_agent_skill_links(self.skills) + + self.assertTrue(any("missing local resource" in error for error in errors)) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/workflows/agent-skills.yml b/.github/workflows/agent-skills.yml index bffe7ea8c51..055b246010f 100644 --- a/.github/workflows/agent-skills.yml +++ b/.github/workflows/agent-skills.yml @@ -5,11 +5,15 @@ on: pull_request: paths: - "skills/**" + - ".github/scripts/validate_agent_skill_links.py" + - ".github/tests/test_validate_agent_skill_links.py" - ".github/workflows/agent-skills.yml" push: branches: ["main"] paths: - "skills/**" + - ".github/scripts/validate_agent_skill_links.py" + - ".github/tests/test_validate_agent_skill_links.py" - ".github/workflows/agent-skills.yml" permissions: @@ -49,22 +53,8 @@ jobs: - name: Validate relative resource links run: | - python - <<'PY' - from pathlib import Path - import re - - missing: list[str] = [] - for path in Path("skills").rglob("*.md"): - for target in re.findall(r"\[[^\]]+\]\(([^)]+)\)", path.read_text(encoding="utf-8")): - if "://" in target or target.startswith(("#", "mailto:")): - continue - relative_path = target.split("#", 1)[0] - if relative_path and not (path.parent / relative_path).resolve().exists(): - missing.append(f"{path}: {target}") - - if missing: - raise SystemExit("\n".join(missing)) - PY + python .github/tests/test_validate_agent_skill_links.py + python .github/scripts/validate_agent_skill_links.py skills - name: Validate GitHub skill publishing env: diff --git a/skills/agent-framework-builder-python/SKILL.md b/skills/agent-framework-builder-python/SKILL.md index ece1a705188..8a0d4508ebf 100644 --- a/skills/agent-framework-builder-python/SKILL.md +++ b/skills/agent-framework-builder-python/SKILL.md @@ -30,9 +30,9 @@ relying on remembered preview-era syntax. with instructions, tools, middleware, context providers, and session handling behind one `run(...)` interface. - A **harness agent** is a regular `Agent` assembled by `create_harness_agent`. It adds an opinionated set of context providers and middleware; it does not introduce a separate agent abstraction. -- A **workflow** coordinates explicit execution paths. Graph workflows and orchestration builders can use multiple - `SupportsAgentRun` implementations as participants or executors and combine them sequentially, concurrently, through - handoffs, group chat, Magentic coordination, or custom routing. +- A **workflow** coordinates explicit execution paths. Graph workflows and most orchestration builders can use multiple + `SupportsAgentRun` implementations or executors. `HandoffBuilder` is the exception: its participants must be concrete + `Agent` instances. ## User interface options @@ -86,13 +86,16 @@ configuration. The endpoint is for MCP clients and returns an error when opened 1. Search Microsoft Learn with `microsoft_docs_search`. 2. Fetch the relevant page with `microsoft_docs_fetch`. 3. Use `microsoft_code_sample_search` when a current implementation example is needed. -4. If Learn MCP is unavailable, use the equivalent `mslearn` command: +4. If Learn MCP is unavailable and a trusted `mslearn` CLI is already installed, use the equivalent command: -| MCP tool | CLI fallback | +| MCP tool | Preinstalled CLI | | --- | --- | -| `microsoft_docs_search(query="...")` | `npx @microsoft/learn-cli search "..."` | -| `microsoft_docs_fetch(url="...")` | `npx @microsoft/learn-cli fetch "..."` | -| `microsoft_code_sample_search(query="...", language="python")` | `npx @microsoft/learn-cli code-search "..." --language python` | +| `microsoft_docs_search(query="...")` | `mslearn search "..."` | +| `microsoft_docs_fetch(url="...")` | `mslearn fetch "..."` | +| `microsoft_code_sample_search(query="...", language="python")` | `mslearn code-search "..." --language python` | + +Do not fetch and execute an unpinned CLI package during a coding session. Use the +[official Microsoft Learn MCP/CLI setup](https://github.com/MicrosoftDocs/mcp) to install or update it intentionally. Canonical starting points: diff --git a/skills/agent-framework-builder-python/references/workflow-builder.md b/skills/agent-framework-builder-python/references/workflow-builder.md index 9cf55bf0851..c121e97c66e 100644 --- a/skills/agent-framework-builder-python/references/workflow-builder.md +++ b/skills/agent-framework-builder-python/references/workflow-builder.md @@ -20,7 +20,8 @@ docstrings for signatures and execution options. - A functional workflow definition must be built before running; each built workflow owns mutable execution state. - Build fresh workflows when executors or wrapped agents hold run-specific state. -- Any `SupportsAgentRun` implementation can participate where an agent executor is accepted. +- Builders that accept `SupportsAgentRun` can use custom or remote agent implementations. `HandoffBuilder` accepts + concrete `Agent` participants only. - Use standard sequential, concurrent, handoff, group-chat, or Magentic builders before recreating those patterns. - Select caller-facing output separately from intermediate progress. - Wrap a workflow as an agent only when callers require the agent contract. Prefer native workflow hosting when the