Skip to content

CI: quiet transient multi-CN startup probes - #451

Open
XuPeng-SH wants to merge 1 commit into
mainfrom
issue-450-startup-readiness
Open

CI: quiet transient multi-CN startup probes#451
XuPeng-SH wants to merge 1 commit into
mainfrom
issue-450-startup-readiness

Conversation

@XuPeng-SH

@XuPeng-SH XuPeng-SH commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Root cause

The standalone multi-CN workflow starts the proxy, then immediately runs an SQL readiness probe. Before a CN is routable, the probe returns "no available CN server". The existing loop intentionally retries, but piping every attempt through tee /dev/stderr surfaced normal startup retries as repeated ERROR lines. In run #28794, the first probe failed at 01:40:34 UTC, CN startup/HAKeeper registration activity followed, the last observed route failure was at 01:40:43, and the SQL probe succeeded at 01:40:44. Run #28800 shows the same transient pattern. The cited BVT failures were separate assertions.

The INFO artifacts do not reveal why the route candidate set remains empty after registration becomes visible (stale proxy snapshot versus admission, work-state, or selector filtering). This change does not alter MatrixOne routing or proxy error semantics.

Change

  • Capture retry output instead of printing every failed probe.
  • Keep the existing bound of 300 attempts and one-second inter-attempt delay.
  • Report elapsed time and successful attempt; if all attempts fail, print the last probe output plus Compose status and recent logs, then fail.

Validation

  • Workflow YAML parsed successfully.
  • Readiness step passes bash -n.
  • In-memory success and timeout-path mocks passed.
  • Live multi-CN Compose was not run locally.

Fixes #450

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] Make transient no-CN startup probes distinguishable in standalone BVT

1 participant