[Bryan Chen (bryanchen-d)]

Caches Playwright's browsers in the product build's test jobs (ADO def 111), so a Playwright CDN outage no longer fails them. This is the ADO half of B11. The GitHub half is #343387.

Stacked on #343325, which adds retryCountOnTaskFailure to the Cache@2 steps. This PR targets that branch so the diff shows only this change. I'll retarget it to main once #343325 merges.

Why

On Oct 21, 4 Windows test jobs in def 111 failed in Download Electron and Playwright (487426, 487513, 487556, 487566). playwright.download.prss.microsoft.com returned ENOTFOUND and the fallback cdn.playwright.dev/builds reset the connection. The step's retryCountOnTaskFailure: 3 was exhausted in under a minute. Those were the only Playwright download failures in 14 days (out of about 9,840 runs of that step), but they come in bursts.

What

There's a new step template, build/azure-pipelines/common/restore-playwright-browsers.yml, used by the Linux, macOS and Windows test step templates right before Download Electron and Playwright. It does two things:

• It sets PLAYWRIGHT_BROWSERS_PATH to $(Build.SourcesDirectory)/.build/ms-playwright.
• It restores that folder with Cache@2 (retryCountOnTaskFailure: 2), keyed on "playwright" | Agent.OS | VSCODE_ARCH | node_modules/playwright/package.json. That's the pinned Playwright version, so the key only changes when Playwright does.

Download Electron and Playwright itself is unchanged. On a hit, playwright install finds the browsers and downloads nothing. On a miss, it downloads as before, and Cache@2's post-job step saves the folder. Electron comes from our Azure feed, so it doesn't need this.

Validation (full def 111, publish off)

1. Cold: 487870. I expect a cache miss, a download as today, and a post-job save.
2. Warm: queued after 1 finishes. I expect a cache hit, and the install step showing no Downloading lines.

Results will be posted here before I ask for signing.

Authored with Copilot

[Bryan Chen (bryanchen-d)]

Def 111 validation (CI mode, publish off): cold, then warm.

	Cold 487870	Warm 487882
Restore Playwright browsers cache	18/18 miss	18/18 hit
Browsers downloaded in Download Electron and Playwright	118	0
Linux x64: restore + download (median)	1.1 s + 17.3 s	4.8 s + 3.6 s
Windows x64: restore + download (median)	1.0 s + 22.5 s	4.9 s + 13.1 s (Electron only)
macOS arm64: restore + download (median)	1.3 s + 30.4 s	38.7 s + 2.7 s

• Cold run: one job per platform saved the cache in its post-job step (Linux 570 MB, Windows 518 MB, macOS 450 MB physical). The other jobs found it already saved and skipped. Its only failures were the 3 known WebM Unit Tests (Electron) failures, because that run predates issue: skip WebM tests that can't open on the shipped Electron #343360 on main.
• Warm run: every test job restored the cache, and playwright install downloaded nothing. That's what removes the dependency on Playwright's CDN, which is failing again tonight on the ADO macOS agents (487876, ERR_TLS_CERT_ALTNAME_INVALID).
• Time: about 10 s faster on Linux and Windows, and about 10 s slower on macOS, where AcesShared restores the roughly 1.2 GB cache more slowly than the CDN download.

Co-authored with Copilot

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

3 open issues

• The cache key uses a file path that’s interpreted relative to the task’s working directory. To make… · New
• The template sets PLAYWRIGHT_BROWSERS_PATH to an absolute path under $(Build.SourcesDirectory),… · New
• Using node -e just to construct and emit the ##vso[task.setvariable ...] line adds quoting… · New

What changed in this PR

Adds an Azure Pipelines step template to cache Playwright browser downloads in product-build test jobs, reducing test flakiness when Playwright CDNs are unavailable.

Changes:

• Introduces a shared restore-playwright-browsers.yml template that sets PLAYWRIGHT_BROWSERS_PATH and restores/saves .build/ms-playwright via Cache@2.
• Wires the template into Linux, macOS, and Windows product-build test step templates immediately before the Playwright install step.

File	Description
build/​azure-pipelines/​common/​restore-playwright-browsers.yml	New template to set browser path and restore Playwright browser cache.
build/​azure-pipelines/​linux/​steps/​product-build-linux-test.yml	Calls the new restore template before Playwright install.
build/​azure-pipelines/​darwin/​steps/​product-build-darwin-test.yml	Calls the new restore template before Playwright install.
build/​azure-pipelines/​win32/​steps/​product-build-win32-test.yml	Calls the new restore template before Playwright install.

_{Review effort: Lite}

[Copilot]

🔵 Needs a closer look

Cold-cache saving and warm-cache reuse still need confirmation from the planned Azure DevOps runs across all three platforms.

No open issues 🎉

3 resolved since last review

• The template sets PLAYWRIGHT_BROWSERS_PATH to an absolute path under $(Build.SourcesDirectory),…
• The cache key uses a file path that’s interpreted relative to the task’s working directory. To make…
• Using node -e just to construct and emit the ##vso[task.setvariable ...] line adds quoting…

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Successful cold-cache saves and warm-cache installs without downloads have not yet been confirmed across all three platforms.

No open issues 🎉

_{Review effort: Balanced}

[vs-code-engineering]

📬 CODENOTIFY

The following users are being notified based on files changed in this PR:

Ladislau Szomoru (@lszomoru)

Matched files:

• build/azure-pipelines/common/restore-playwright-browsers.yml
• build/azure-pipelines/darwin/steps/product-build-darwin-test.yml
• build/azure-pipelines/linux/steps/product-build-linux-test.yml
• build/azure-pipelines/win32/steps/product-build-win32-test.yml