google · kazunori279 · Aug 7, 2026 · Aug 7, 2026 · Aug 7, 2026 · Aug 7, 2026
diff --git a/docs/evaluate/user-sim.md b/docs/evaluate/user-sim.md
@@ -294,3 +294,69 @@ The `--user_simulation_config_file` expects a JSON file matching the `Conversati
 *   **`generation_instruction`** (optional): A natural language prompt guiding the specific types of scenarios or goals you want to test.
 *   **`environment_context`** (optional): Context describing the backend data or state accessible to the agent's tools. This helps the generator create queries that are grounded in realistic data (e.g., valid device IDs).
 *   **`model_name`** (required): The Gemini model used for generation (e.g., `gemini-flash-latest`).
+
+## Audio user simulation (live agents)
+
+The user simulator is independent of whether the agent under test is a live
+(voice) agent, so the same `ConversationScenario` (or a fixed conversation) can
+drive both text and live evals. For live agents, the simulated user's turns can
+be synthesized to **audio** and streamed to the agent.
+
+This is configured with the `llm_audio` user simulator in your eval config
+(`test_config.json`). It wraps the standard text simulator and converts each
+generated user turn to audio using a text-to-speech model. By default it uses
+Google Cloud Text-to-Speech (`cloud_tts`); a Gemini TTS model name may be used
+instead.
+
+```json
+{
+  "criteria": {
+    "tool_trajectory_avg_score": 1.0,
+    "response_match_score": 0.5
+  },
+  "live_model_config": {
+    "timeout_seconds": 300
+  },
+  "user_simulator_config": {
+    "type": "llm_audio",
+    "model": "gemini-2.5-flash",
+    "audio_model": "cloud_tts",
+    "audio_model_configuration": {
+      "speech_config": {
+        "voice_config": {
+          "prebuilt_voice_config": { "voice_name": "en-US-Studio-O" }
+        },
+        "language_code": "en-US"
+      }
+    },
+    "include_text_with_audio": true
+  }
+}
+```
+
+Key fields:
+
+*   `type`: `"llm_audio"` selects the audio user simulator.
+*   `audio_model`: `"cloud_tts"` for Google Cloud Text-to-Speech, or a Gemini
+    TTS model name (e.g. `"gemini-2.5-flash-preview-tts"`).
+*   `audio_model_configuration.speech_config`: Selects the voice and language.
+*   `include_text_with_audio`: Whether the user turn also carries the text part
+    alongside the generated audio.
+
+!!! note "Live models require live inference"
+
+    Evaluating a live agent requires live (bidirectional streaming) inference,
+    which is **not** the default. Enable it by adding a `live_model_config`
+    block to your config file. Live API models (e.g. `gemini-*-live-*`) are not
+    served over the unary `generateContent` endpoint that non-live eval uses, so
+    running them without live mode fails.
+
+    `use_live` is an internal field set from `live_model_config`; putting it in
+    a config file has no effect.
+
+    Using `cloud_tts` requires the `google-cloud-texttospeech` package (included
+    in the `google-adk[eval]` extra) and access to the Cloud Text-to-Speech API.
+
+See the sample at
+[`contributing/samples/live/live_non_blocking_tool_agent`](https://github.com/google/adk-python/tree/main/contributing/samples/live/live_non_blocking_tool_agent)
+for a complete, runnable live eval configuration.
diff --git a/docs/get-started/about.md b/docs/get-started/about.md
@@ -68,7 +68,7 @@ agentic applications:
    UI for running agents, inspecting execution steps (events, state changes),
    debugging interactions, and visualizing agent definitions.
 5. **Native Streaming Support:** Build real-time, interactive experiences with
-   [ADK Gemini Live API Toolkit](../live/index.md) that provides native support for bidirectional
+   [live and voice agents](../live/index.md) that provide native support for bidirectional
    streaming (text and audio). This integrates seamlessly with underlying
    capabilities like the [Gemini Live API for the Gemini Developer API](https://ai.google.dev/gemini-api/docs/live)
    (or for

diff --git a/...ets/adk-streaming-guide-quota-console.png → ...ets/adk-streaming-guide-quota-console.png b/...ets/adk-streaming-guide-quota-console.png → ...ets/adk-streaming-guide-quota-console.png
diff --git a/docs/live/audio-video.md b/docs/live/audio-video.md
@@ -0,0 +1,119 @@
+# Audio and video
+
+<div class="language-support-tag">
+    <span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v0.1.0</span>
+</div>
+
+Audio and video are what make a live agent feel live, and they are where the exact formats
+matter. The Live API expects specific PCM sample rates for audio, and images and video
+frames go through a different send method than text.
+
+**ADK does not convert media for you.** Getting the sample rate, encoding, and MIME type
+right is your responsibility, and the wrong format produces silence, noise, or a connection
+error rather than a helpful message. What follows is that contract.
+
+For the models that support these modalities, see [Supported models](models.md). For voices,
+transcription, and turn detection, see [Configuration](configuration.md). For a client that
+already implements all of this, run your agent in `adk web`; to write your own, see
+[Build a custom server](custom-server.md#connect-a-client).
+
+## Audio input
+
+Send microphone audio as raw bytes through
+[`send_realtime()`](sessions.md#liverequestqueue). The bytes must already be in the format
+the Live API expects — ADK passes them straight through:
+
+| Property | Value |
+|----------|-------|
+| Encoding | 16-bit PCM, signed, little-endian |
+| Sample rate | 16,000 Hz (16 kHz) |
+| Channels | Mono |
+| MIME type | `audio/pcm;rate=16000` |
+
+```python
+from google.genai import types
+
+live_request_queue.send_realtime(
+    types.Blob(mime_type="audio/pcm;rate=16000", data=audio_data)
+)
+```
+
+Stream audio in small chunks for low latency. `LiveRequestQueue` forwards each chunk
+promptly without coalescing, so the chunk size you send is the granularity the model
+receives:
+
+- **Ultra-low latency** (real-time conversation): 10-20 ms per chunk.
+- **Balanced** (recommended): 50-100 ms per chunk. At 16 kHz, 100 ms is
+  `16000 × 0.1 × 2 = 3200` bytes.
+- **Lower overhead**: 100-200 ms per chunk.
+
+Use a consistent chunk size for the session, and do not wait for a model response before
+sending the next chunk — the model processes audio continuously, not turn by turn. With
+[voice activity detection](configuration.md#voice-activity-detection-vad) on (the default),
+stream continuously and let the API detect speech; send
+[activity signals](sessions.md#liverequestqueue) only when you disable VAD.
+
+## Audio output
+
+With `response_modalities=["AUDIO"]` (the live default), the model returns audio as
+`inline_data` parts on the event stream:
+
+| Property | Value |
+|----------|-------|
+| Encoding | 16-bit PCM, signed, little-endian |
+| Sample rate | 24,000 Hz (24 kHz) — note this differs from the 16 kHz input rate |
+| Channels | Mono |
+| MIME type | `audio/pcm;rate=24000` |
+
+```python
+async for event in runner.run_live(...):
+    if event.content and event.content.parts:
+        for part in event.content.parts:
+            if part.inline_data and part.inline_data.mime_type.startswith("audio/pcm"):
+                await play_audio(part.inline_data.data)  # raw 24 kHz PCM bytes
+```
+
+The bytes arrive ready to play; no decoding is needed on your side. The Live API transmits
+audio as base64 over the wire, but `google.genai` decodes it for you, so `part.inline_data.data`
+is already `bytes`. For which events carry audio and how they interleave with transcription,
+see [Events](events.md#audio). To persist audio to the artifact service, set
+[`save_live_blob=True`](configuration.md#save_live_blob).
+
+## Images and video
+
+Images and video are sent as individual JPEG frames through the same
+[`send_realtime()`](sessions.md#liverequestqueue) method as audio. There is no video codec:
+a video stream is a sequence of still frames, each sent as its own blob.
+
+| Property | Value |
+|----------|-------|
+| Format | JPEG (`image/jpeg`) |
+| Frame rate | ~1 frame per second (recommended maximum) |
+| Resolution | 768×768 pixels (recommended) |
+
+```python
+from google.genai import types
+
+live_request_queue.send_realtime(
+    types.Blob(mime_type="image/jpeg", data=jpeg_bytes)
+)
+```
+
+At ~1 FPS the model can see what the user is pointing a camera at or discussing, but not
+anything motion-dependent. Action recognition, sports analysis, and motion tracking need
+temporal resolution this approach does not provide.
+
+In the [Shopper's Concierge demo](https://youtu.be/LwHPYyw7u6U?si=lG9gl9aSIuu-F4ME&t=40),
+the app sends a user-uploaded image with `send_realtime()`; the agent recognizes the context
+and searches an e-commerce catalog for matching items.
+
+<div class="video-grid">
+  <div class="video-item">
+    <div class="video-container">
+<iframe width="560" height="315" src="https://www.youtube.com/embed/LwHPYyw7u6U?si=lG9gl9aSIuu-F4ME&amp;start=40" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
+    </div>
+  </div>
+</div>
+
+To feed a live video stream into a tool so the agent can react to frames as they arrive, see
+[Streaming tools](tools.md#streaming-tools).