Skip to content

Give the slow-runner test waits the same headroom as everything else - #193

Merged
arzafran merged 1 commit into
mainfrom
fix/scale-compat-test-timeouts
Jul 29, 2026
Merged

Give the slow-runner test waits the same headroom as everything else#193
arzafran merged 1 commit into
mainfrom
fix/scale-compat-test-timeouts

Conversation

@arzafran

Copy link
Copy Markdown
Member

What this does

The suite already knows CI machines are slower and stretches its wait budgets 4x when it runs there (ciScale, TabManagerUnitTests.swift:22). That stretch was threaded into most of the suite but never into the window and overlay tests, which still wait a fixed few seconds no matter how loaded the machine is.

Those are the tests that keep failing the macOS Compatibility job on the macos-15 runner, and a different handful fails each run — 8, then 8, then 16, then 19 across today's runs. That is what losing a scheduler race looks like, not something being broken. The strongest evidence: TerminalWindowPortalLifecycleTests, which sits directly below the failing class in the same file, already scales its spins by ciScale and has stayed green throughout.

Both wait helpers now scale the same way, so their tests get the headroom the rest of the suite has had.

Summary

  • GhosttySurfaceOverlayTests.waitUntil (TerminalAndGhosttyTests.swift:2089) now waits timeout * ciScale. Scaling inside the helper rather than at each call site keeps all 11 of them consistent — those sites pass raw 3.0 and 10.0 literals, which become 12s and 40s under CI.
  • WorkspaceSplitWorkingDirectoryTests.waitForCondition (WorkspaceUnitTests.swift:2032) scales its deadline the same way. It is a local copy of the helper in TabManagerUnitTests.swift, which already scaled; this one never did.
  • No production code changes. No call sites touched.

What this deliberately does NOT change

The fixed-duration RunLoop.current.run(until:) pumps in BrowserPanelTests.swift (~2093-2103) and the waits in TerminalControllerSocketSecurityTests.swift also have no scaling. They are left alone on purpose so the result of this change stays interpretable — if compat goes green, the overlay helpers were the dominant cause; if specific non-overlay tests still fail, those are the remaining suspects and get their own change.

Ruled out

  • Not a macos-15 behavioral difference — no OS-conditional code in these paths.
  • Not the thread-starvation bug fixed in Stop subprocess tests timing out when the machine is busy #191. That was file-scoped to WorkspaceRemoteConnectionTests.swift, and CLINotifyProcessIntegrationTests.testSSHBootstrapStartupCommandPassesRemoteInstallScriptAsSingleSSHCommand — which appeared in earlier failure lists — was an observation from before that fix landed.
  • Not parallel-worker contention — this job sets PROGRAMA_UNIT_TEST_SCOPE: serial, which passes -parallel-testing-enabled NO.

Test plan

  • xcodebuild -scheme programa-unit build-for-testing succeeds
  • compat-tests (macos-15) passes on this PR
  • It passes on a second run — these tests have passed by luck before, so a single green run does not distinguish a fix from a lucky one

…else

The test suite already knows CI machines are slower and stretches its
waiting times four times longer when it runs there. That stretch was
added to most of the suite but never to the window and overlay tests,
which still waited for a fixed few seconds no matter how loaded the
machine was.

Those are the tests that have been failing on the older macOS runner,
and a different handful failed each time, which is what losing a race
looks like rather than something being broken. The class sitting right
next to them in the same file already had the stretch and has been fine.

Both waiting helpers now stretch the same way, so all of their tests get
the same headroom the rest of the suite has had.
@arzafran
arzafran merged commit 6655461 into main Jul 29, 2026
9 of 10 checks passed
@arzafran
arzafran deleted the fix/scale-compat-test-timeouts branch July 29, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant