Fix kern subsetting regression for fonts with >64KB kern tables

[wnvko-msft]

Validate format 0 kern pair data against the uint32 kern table-directory length instead of the uint16 subtable length field, which wraps for subtables over 64KB and wrongly rejected valid fonts during XPS/PDF export and printing.

Co-authored-by: Copilot

Fixes #11834 and #11836

Description

During font subsetting (XPS/print/PDF export), AdjustKernFormat0 validated the format 0 kern pair data against the kern subtable length header field. Per the OpenType kern spec that field is a uint16, so for any kern subtable larger than 64KB it wraps to a value far smaller than the real size. The bounds check then sees the computed pair-array size exceed the wrapped length and rejects the font, failing the subset/export even though the font is valid.

This change validates against the uint32 kern table-directory length (TTTableLength(pOutputBufferInfo, KERN_TAG)), which is a genuine 32-bit field that correctly describes tables over 64KB and was already validated against the buffer by CopyTableOver earlier in the same path.

Customer Impact

Applications that print or export documents using common fonts with large kern tables (for example the Calibri family) fail during XPS/PDF export and printing. Some scenarios surface a FileFormatException, and in certain apps the failure is
unhandled and terminates the process. Without this fix, affected documents cannot be printed or exported without switching fonts or disabling the protection via the opt-out AppContext switch.

Regression

Yes. This is a regression introduced by the recent font-parsing hardening in the TrueType subsetter. Documents that previously printed/exported correctly began failing for fonts whose kern table exceeds 64KB.

Testing

Verified with a WPF app that exports every installed font to XPS:

• Before the fix: fonts with >64KB kern tables (Calibri, Cambria, Constantia, Corbel) fail to subset/export.
• After the fix: all installed fonts export successfully.
• Confirmed the existing opt-out switch Switch.MS.Internal.TtfDelta.DisableCmapAndSbitOverflowProtection still behaves as before (reverts to the pre-hardening code path).

Risk

Low. The change is a single comparison - it swaps the wrapping uint16 subtable length for the uint32 kern table-directory length that is already verified against the buffer earlier in the same code path. The ULongMult32 overflow guard and all per-read/-write bounds checks (CheckInOffset/CheckOutOffset) are untouched, so the hardening remains fully in effect; the fix only stops valid large-kern fonts from being wrongly rejected.

Microsoft Reviewers: Open in CodeFlow

[Copilot]

Pull request overview

This PR addresses a regression in the TrueType subsetter’s kern table handling that caused valid fonts with kern subtables larger than 64KB (where the subtable’s uint16 length wraps) to be rejected during XPS/PDF export and printing. The change updates the bounds validation for format 0 kern pair data to use the kern table-directory uint32 length instead of the wrapping uint16 subtable length.

Changes:

• Update AdjustKernFormat0 bounds validation to compare pair-array size against TTTableLength(..., KERN_TAG) rather than KernSubHeader.length.
• Add an explanatory comment describing the uint16 wrap issue for large kern subtables.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.