Skip to content

test: bound and retry apt in the WSL smoke setup - #345598

Merged
Bryan Chen (bryanchen-d) merged 3 commits into
mainfrom
copilot/vscode/wsl-smoke-apt-retry
Nov 2, 2026
Merged

Bryan Chen (bryanchen-d) merged 3 commits into
mainfrom
copilot/vscode/wsl-smoke-apt-retry

Conversation

@bryanchen-d

@bryanchen-d Bryan Chen (bryanchen-d) commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes a flaky Prepare WSL Dev Container smoke tests step on Windows (PR CI Windows / Electron-Smoke (shard 1/3) and def 111 Windows smoke).

Mechanism

  • The step installs docker.io and a few packages inside the WSL distro from the Azure Ubuntu mirror (test: use the Azure Ubuntu mirror in the WSL smoke setup #343768). Normally the whole package download takes 2-17 seconds (24 green PR CI jobs sampled over the last 48h).
  • Sometimes a connection to the mirror crawls for minutes. apt has no minimum transfer speed, so the download runs past Invoke-Wsl's 300 second timeout. That happened in 2 PR CI runs in the last 48h, both stuck in apt-get install (containerd 28 MB took about 3 minutes in run 41482962880, with Ign: retries in run 41251126966).
  • When the mirror is unreachable, apt-get update only warns (W: Failed to fetch … Could not connect to azure.archive.ubuntu.com) and exits 0. The install then fails with E: Unable to locate package docker.io (def 111 main CI 491631).

Fix

  • apt-get update --error-on=any, so a failed index fetch fails the attempt instead of being ignored.
  • Acquire::http::Timeout "30": a connection that stops sending data fails after 30s, and apt retries it (Acquire::Retries "5"). There is no wall clock on the download itself, so a slow but progressing download gets the whole step budget. The download (--download-only) and a --error-on=any index update (bounded at 45s per attempt) are each retried up to 3 times on error. The final apt-get install installs from the local cache.
  • That apt step's Invoke-Wsl budget goes from 300 to 720 seconds to fit the bounded retries. A healthy run still takes seconds.
  • This PR's own first CI run hit the slow case (job 125428287654). The indexes downloaded at 8 MB/s, but packages came at about 250 KB/s on every connection, so fresh connections alone don't help. At that rate, 80 MB needs about 320s: more than the old 300s, but within the new budget. The first version of this PR used a 100s cap per attempt, and that kept cutting the crawling containerd and docker.io downloads off mid-file.

Validation

  • PowerShell parses the script, and the expanded bash block passes bash -n.
  • def 111 Windows CI run 491857 on a proof branch (103334e962) that also refuses the first index update and throttles the first package download to 64 KB/s. Prepare WSL Dev Container smoke tests succeeded. With --error-on=any, the refused update failed attempt 1 instead of warning, and attempt 2 fetched the indexes in 5s. The throttled download ran past 100s on attempt 1, and attempt 2 resumed and finished the remaining 73.4 MB in 1s. Docker then started, and the rest of the setup passed.
  • def 111 Windows CI run 491868 on the proof branch for the current design (5904161ba6). It refuses the first index update and throttles the whole package download to 256 KB/s. The step succeeded: attempt 1 of the update failed and attempt 2 passed, then Fetched 79.9 MB in 4min 5s (326 kB/s) completed, which would not fit in the old 300s budget.
  • PR CI on the current head: Windows / Electron-Smoke (shard 1/3) passed on the healthy path.

Authored with Copilot

Copilot AI balanced review requested due to automatic review settings November 2, 2026 17:14
@bryanchen-d

Copy link
Copy Markdown
Collaborator Author

Ladislau Szomoru (@lszomoru) Christof Marti (@chrmarti) FYI. This bounds and retries the apt step in wslDevContainer.ps1 that #343768 moved to the Azure mirror. The mirror still crawls or refuses connections now and then. #345511 adds Kerberos and squid packages to the same apt-get install line, which means more to download, so the two PRs will conflict on that one line. Whichever lands second only needs to add its packages to $aptPackages.

Co-authored with Copilot

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The WSL Dev Container smoke-test setup changes have no active review issues.

No open issues 🎉

What changed in this PR

Improves reliability of the WSL Dev Container smoke-test setup by bounding and retrying apt operations when the Azure Ubuntu mirror stalls or fails.

Changes:

  • Makes failed package-index fetches fail the update attempt.
  • Retries index updates and download-only installs up to three times, with respective 60- and 100-second limits.
  • Raises the WSL command timeout from 300 to 600 seconds to accommodate retries before installation from the package cache.
File Description
test/​smoke/​wslDevContainer.ps1 Adds bounded apt retries, separates downloading from installation, and extends the setup timeout.

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the WSL Dev Container apt timeout and retry changes.

No open issues 🎉

Review effort: Balanced

The Azure Ubuntu mirror usually serves the WSL smoke packages in 2-17
seconds, but a connection to it can fail or crawl for minutes. apt has
no minimum transfer speed, so a crawling download ran past the 300
second command timeout (2 PR CI failures in 48h), and apt-get update
only warns when it can't fetch the indexes, so the install then failed
to find docker.io (main CI 491631).

Bound the index update (now failing on any fetch error) and the package
download, and retry each on fresh connections. A retried download
resumes partial files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… attempts

This PR's own CI hit the slow case: the indexes came at 8 MB/s, but
packages came at about 250 KB/s on every connection, so fresh connections
alone don't help. Each bounded attempt keeps completed packages and
resumes the partial one, so allow 5 download attempts and a 720 second
budget for the step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… budget

Instead of a hard per-attempt wall clock, which restarted a download
that was progressing at about 250 KB/s, fail only a connection that
stops sending data (Acquire::http::Timeout 30) and retry the download
on error.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the WSL dev-container smoke setup changes.

No open issues 🎉

Review effort: Balanced

@bryanchen-d
Bryan Chen (bryanchen-d) marked this pull request as ready for review November 2, 2026 18:25
@bryanchen-d
Bryan Chen (bryanchen-d) enabled auto-merge (squash) November 2, 2026 18:25
@bryanchen-d
Bryan Chen (bryanchen-d) merged commit 4bf0834 into main Nov 2, 2026
56 checks passed
@bryanchen-d
Bryan Chen (bryanchen-d) deleted the copilot/vscode/wsl-smoke-apt-retry branch November 2, 2026 18:37
@vs-code-engineering vs-code-engineering Bot added this to the 1.146.0 milestone Nov 2, 2026
Aaron Munger (amunger) pushed a commit that referenced this pull request Nov 5, 2026
* test: bound and retry apt in the WSL smoke setup

The Azure Ubuntu mirror usually serves the WSL smoke packages in 2-17
seconds, but a connection to it can fail or crawl for minutes. apt has
no minimum transfer speed, so a crawling download ran past the 300
second command timeout (2 PR CI failures in 48h), and apt-get update
only warns when it can't fetch the indexes, so the install then failed
to find docker.io (main CI 491631).

Bound the index update (now failing on any fetch error) and the package
download, and retry each on fresh connections. A retried download
resumes partial files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: allow a slow mirror to finish the WSL smoke apt download across attempts

This PR's own CI hit the slow case: the indexes came at 8 MB/s, but
packages came at about 250 KB/s on every connection, so fresh connections
alone don't help. Each bounded attempt keeps completed packages and
resumes the partial one, so allow 5 download attempts and a 720 second
budget for the step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: let a slow but progressing WSL smoke apt download use the whole budget

Instead of a hard per-attempt wall clock, which restarted a download
that was progressing at about 250 KB/s, fail only a connection that
stops sending data (Acquire::http::Timeout 30) and retry the download
on error.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Aaron Munger (amunger) added a commit that referenced this pull request Nov 5, 2026
* agentHost: retain Copilot identity for edit-survival telemetry (#345691)

Capture the originating account tracking ID for file tools and preserve it in host edit-source state until delayed telemetry emission. Keep different accounts separate without exposing identity in protocol markers or source keys.

Fixes microsoft/vscode-internalbacklog#10569

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Bump version to 1.145.1

Update Copilot Chat's VS Code engine requirement while retaining its 0.73.0 package version.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Revert "Bump version to 1.145.1"

This reverts commit 2dac9e3.

* test: bound and retry apt in the WSL smoke setup (#345598)

* test: bound and retry apt in the WSL smoke setup

The Azure Ubuntu mirror usually serves the WSL smoke packages in 2-17
seconds, but a connection to it can fail or crawl for minutes. apt has
no minimum transfer speed, so a crawling download ran past the 300
second command timeout (2 PR CI failures in 48h), and apt-get update
only warns when it can't fetch the indexes, so the install then failed
to find docker.io (main CI 491631).

Bound the index update (now failing on any fetch error) and the package
download, and retry each on fresh connections. A retried download
resumes partial files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: allow a slow mirror to finish the WSL smoke apt download across attempts

This PR's own CI hit the slow case: the indexes came at 8 MB/s, but
packages came at about 250 KB/s on every connection, so fresh connections
alone don't help. Each bounded attempt keeps completed packages and
resumes the partial one, so allow 5 download attempts and a 720 second
budget for the step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: let a slow but progressing WSL smoke apt download use the whole budget

Instead of a hard per-attempt wall clock, which restarted a download
that was progressing at about 250 KB/s, fail only a connection that
stops sending data (Acquire::http::Timeout 30) and retry the download
on error.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Bryan Chen <41454397+bryanchen-d@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants