[None][doc] Clarify KV cache host offload limits

[zcxGGmu]

Summary

• Clarify that KV cache host offloading is for reuse/preemption, not extending the active context length.
• Document that blocks for an actively processed request must be resident in GPU memory before attention can use them.
• Point users toward GPU KV budget, KV dtype/model size, attention-window features, or larger GPUs for longer active contexts.

Test Plan

• Static doc smoke check for the new host-offload limitation text
• git diff --check
• Not run: full Sphinx build (python -m sphinx --version fails locally with No module named sphinx)

Refs #7281

Dev Engineer Review

• Updated docs/source/features/kvcache.md.
• Added the standard 2026 NVIDIA SPDX copyright and license header.
• Clarified that host-memory KV offloading supports cache reuse and preemption.
• Clarified that active attention requires blocks to be resident in GPU memory.
• Documented alternatives for longer active contexts.
• No code, API, configuration, or public entity changes.
• Documentation checks and git diff --check passed. A full Sphinx build was not run because Sphinx was unavailable.

QA Engineer Review

No test changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5962e93f-3efd-4914-ab10-0853a2bdc029

📥 Commits

Reviewing files that changed from the base of the PR and between 7900d55 and c3544e1.

📒 Files selected for processing (1)

• docs/source/features/kvcache.md

---

Walkthrough

The KV cache documentation adds licensing metadata and explains that host-memory offloading supports reuse and preemption but does not increase the active GPU-resident context length.

Changes

KV cache documentation

Layer / File(s)	Summary
Clarify offloading behavior docs/source/features/kvcache.md	Adds the copyright and license header. Documents GPU residency requirements for active requests, reuse of evicted or paused blocks, and alternatives for larger active contexts.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Suggested reviewers: brnguyen2

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies this as documentation that clarifies KV cache host offload limits.
Description check	✅ Passed	The description explains the change, test coverage, unavailable Sphinx validation, and issue reference, but omits the template checklist.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

docs/source/features/kvcache.md (1)

165-171: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use established terms for reuse.

Line [166] uses a long construction, and “later onboarding” in Line [169] is not defined in this section. Use “active request” and “later reuse” consistently.

Proposed wording

-Host offloading is a reuse and preemption mechanism, not a way to extend the
-active context length beyond what the GPU KV cache can hold for a request that
-is currently running.
+Host offloading supports reuse and preemption. It does not extend the active
+context length beyond the GPU KV cache capacity of an active request.
...
-evicted or paused blocks available for later onboarding, which avoids
+evicted or paused blocks available for later reuse, which avoids

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/source/features/kvcache.md` around lines 165 - 171, Update the
host-offloading explanation near the active-request discussion to use the
established terms “active request” and “later reuse” consistently. Replace the
undefined “later onboarding” wording and simplify the long reuse description
without changing the distinction between active GPU-resident blocks and evicted
CPU-resident blocks.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/features/kvcache.md`:
- Around line 165-175: Add the required NVIDIA copyright header to
docs/source/features/kvcache.md, preserving the existing documentation content
and formatting.

---

Nitpick comments:
In `@docs/source/features/kvcache.md`:
- Around line 165-171: Update the host-offloading explanation near the
active-request discussion to use the established terms “active request” and
“later reuse” consistently. Replace the undefined “later onboarding” wording and
simplify the long reuse description without changing the distinction between
active GPU-resident blocks and evicted CPU-resident blocks.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: aff68332-342b-4c9d-b235-fc0fa990bd7f

📥 Commits

Reviewing files that changed from the base of the PR and between f547987 and 7900d55.

📒 Files selected for processing (1)

• docs/source/features/kvcache.md

[thorjohnsen]

This is a correct statement of the capabilities of the current system and the intended use of the offloading feature. For the record, we tried attending to blocks in host memory as a way to expand the capacity of the system beyond what GPU memory alone could provide, but throughput as deemed too low. This was an undocumented feature for some time, but was removed some time ago. Current system does not have the ability to attend directly to blocks in host memory.

[zcxGGmu]

Thanks for the review — I pushed c3544e168 addressing the feedback.

Changes made:

• Added the standard 2026 NVIDIA SPDX copyright/license header to docs/source/features/kvcache.md.
• Preserved the existing document title and content.

Validation:

• python static documentation guard checks — passed for header, title, and balanced code fences.
• git diff --check — passed.
• Added-line secret scan — passed.

Note: Full Sphinx docs build was not run locally because sphinx is not installed in this environment.