From 2dc8fd727c46a179e8fa9c661bf8b2c4063dc35b Mon Sep 17 00:00:00 2001 From: Jurie Smit Date: Thu, 13 Aug 2026 02:30:05 +0200 Subject: [PATCH] ci(engine): let the ADR 17 sluice metadata tests fail the build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit engine/tests/test_sluice_request_metadata.py (92 tests) covers the ADR 17 {app, agent} request-metadata contract, including the case where the async ai.core stack is rerouted to the gateway by OPENAI_BASE_URL / ANTHROPIC_BASE_URL alone. It lives under tests/, so the only step that ran it was "Run tests with coverage" — which carries continue-on-error for the 75 pre-existing failures in that tree, and therefore masked this suite too. Extend the existing ratchet pattern rather than dropping continue-on-error wholesale: a sibling enforcing step re-runs the suite (~1 min) so it gates for real, matching what "Enforce quality_engine tests" already does. The suite is green at 92 passed. Verified it gates rather than merely runs: reverting the guard to inspect config only (dropping the SDK env-var arm) fails 10 tests and exits 1. `if: !cancelled()` so a failure in the quality_engine ratchet reports alongside this one instead of short-circuiting it. Refs baton 0525aefd-6903-4da1-9d3c-7541837e2d45 Co-Authored-By: Claude Opus 5 --- .github/workflows/ci.yml | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 06f6e07..0455bbe 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -52,7 +52,7 @@ jobs: # Runs both test trees for coverage. Still continue-on-error because tests/ has # 75 pre-existing failures and 56 errors that this step has been masking; making # it gate today would just turn master red. Tracked separately — remove this line - # once tests/ is clean, and delete the ratchet step below when you do. + # once tests/ is clean, and delete the ratchet steps below when you do. - name: Run tests with coverage continue-on-error: true run: | @@ -66,6 +66,24 @@ jobs: run: | poetry run pytest codeflow_engine/actions/quality_engine/__tests__ + # Ratchet: the ADR 17 request-metadata suite is green (92 passed), so it gates + # too. Unlike quality_engine this one does live under tests/, so the step above + # already collects it — and then masks every failure, which left a contract other + # repos depend on effectively untested here. + # What it holds: every Sluice-bound request carries {app, agent}, agent stays + # inside the closed label set, and no provider stack reaches the gateway + # untagged. That last part is why config inspection is not enough — the async + # ai.core stack never passes base_url, so OPENAI_BASE_URL / ANTHROPIC_BASE_URL + # alone can reroute it to Sluice with no metadata block and nothing in a diff to + # notice. Sluice's ADR 17 is heading for stage-3 enforcement (untagged requests + # get HTTP 400), so a regression here becomes a production failure, not a lint. + # Re-runs those 92 (~1 min). `!cancelled()` so a failure in the ratchet above + # reports alongside this one instead of hiding it. + - name: Enforce ADR 17 sluice request-metadata tests + if: '!cancelled()' + run: | + poetry run pytest tests/test_sluice_request_metadata.py + - name: Upload coverage to Codecov uses: codecov/codecov-action@v4 with: