Skip to content

test(mutation): add a bounded Stryker harness for the merge-decision code - #572

Open
hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-mutation-harness
Open

hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-mutation-harness

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds the bounded, repeatable mutation check that #554's "Strengthen assertions and the test harness" section asks for ("Establish bounded, repeatable mutation checks for selected high-risk behavior, with reproducible commands and reviewed equivalent-mutant exclusions. A repository-wide mutation gate is not required.").

Files / cluster claimed: stryker.config.mjs, vitest.mutation.config.mjs, package.json (test:mutation script + @stryker-mutator/core, @stryker-mutator/vitest-runner devDependencies), package-lock.json, .gitignore, docs/contributing.md (new "Mutation checks" section appended after the phantom-branches section — disjoint from #556's table-row edit), __tests__/utils/labelMatch.test.ts (matchesLabelPattern middle-part overlap). No src/ or dist/ change.

First run (main @ e3119ad, Stryker 10.0.0, Vitest 5.0.4, Node 26)

File Mutants Killed Survived
mergeGate.ts 24 24 0
labelMatch.ts 47 46 → 47 1 → 0
hold.ts 48 42 6
duration.ts 32 30 2

11 m 36 s on 2 workers. The one real survivor, labelMatch.ts:35 (at = found - part.length: a middle *-separated part could reuse characters of the part before it, so *ab*b* matched ab), is pinned by the new does not let a middle part reuse characters of the part before it case. The eight remaining survivors are reviewed in the docs table (two equivalent in duration.ts; six in hold.ts whose only observable is @actions/core's own error text, an issue_comment payload GitHub never sends, or a core.debug string).

Not in this PR

A scheduled CI lane for this command would be a .github/workflows/** change, which this agent's token cannot push; #554 says a repository-wide gate is not required, so none is proposed here.

Validation

  • npm run test:mutation (twice: before the shim → 123 survived / 0 tests per runtime mutant; after → 142 killed; incremental re-run on labelMatch.ts after the new test → 47/47 killed)
  • npx vitest run __tests__/utils/labelMatch.test.ts → 24 passed
  • npx eslint stryker.config.mjs vitest.mutation.config.mjs __tests__/utils/labelMatch.test.ts docs/contributing.md → clean
  • package-lock.json: 169 added entries, 0 removed, 4 @inquirer/* patch bumps pulled in by Stryker

Related Issue

Refs #554 — this lands the "bounded, repeatable mutation checks" box of the harness section; the tracker's CI-enforcement, release-delivery, remaining harness boxes (#569/#571 cover the unexpected-traffic ones) and routing sections stay open and are owned elsewhere.


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@mrbobbytables

…code

`npm run test:mutation` runs StrykerJS over src/utils/mergeGate.ts,
src/utils/labelMatch.ts, src/labels/hold.ts and src/utils/duration.ts
through vitest.mutation.config.mjs, which leaves out the bundle harness
(it runs the committed dist/index.js, which no mutation of src/ can
reach) and turns coverage off. Reports and the incremental file land in
reports/mutation/, which is ignored along with .stryker-tmp.

The Vitest config also carries a shim for stryker-mutator/stryker-js#6210:
vitest-runner 10.0.0 selects a mutant's tests with a testNamePattern of
space-joined names while Vitest 5 matches the " > "-joined chain, so
without it every mutant inside a describe() is reported Survived with 0
tests run.

First run: 151 mutants, 142 killed. The one real survivor,
labelMatch.ts:35 (`at = found - part.length` let a middle part reuse
the characters of the part before it), is pinned in labelMatch.test.ts;
the eight remaining survivors are reviewed in docs/contributing.md.

Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive
hivecommons-hive Bot requested a review from jpmcb as a code owner October 23, 2026 07:49
@hivecommons-hive hivecommons-hive Bot added the hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 23, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Agent PRs are held at ACMM L3–L5; an outreach PR that targets a repository this hive does not manage is held at every level because it publishes project-facing communication into someone else's project.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author
24dfbaf

ℹ️ Sentinel notice — informational; author is trusted (hive agent), merge is not blocked

Hive flagged this PR (author @hivecommons-hive[bot], head 24dfbaf13fdc) because it matches behaviors that can override security controls, escalate privileges or damage the codebase. This is a heuristic, not an accusation — a maintainer should confirm the change is intended before it merges.

  • sensitive_path — changes 2 sensitive path(s) (Touches a sensitive path (OWNERS, workflows, policies, hive config, security docs, …))
    • package-lock.json
    • package.json

Hive did not add the sentinel-alert label because this author is trusted. The finding remains recorded for audit and the dashboard Security tab; set sentinel.trusted_authors_block: true to require the blocking label for trusted authors too.

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

/kind cleanup


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5q9t | SHA: e3119ad

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

@github-actions github-actions Bot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. and removed needs-kind labels Oct 23, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the tree at merge-base (no src/ or dist/ change; config, tests, docs, lockfile only). This looks correct to me.

Verified:

  • New test kills the real survivor. Traced matchesLabelPattern (src/utils/labelMatch.ts:35) by hand: under the mutant at = found - part.length, '*ab*b*' vs 'ab' gives indexOf('b', -2) → 1 → true; the real code returns false. All four new assertions in __tests__/utils/labelMatch.test.ts:44-50 distinguish the two.
  • Shim mechanism is real. Vitest 5 matches testNamePattern against the " > "-joined fullTestName while @stryker-mutator/vitest-runner 10.0.0 assigns a space-joined pattern (upstream stryker-mutator/stryker-js#6210); without the rewrite in vitest.mutation.config.mjs:24-38 every mutant inside a describe() survives with 0 tests. The Object.defineProperty setter intercepts Stryker's per-mutant assignment correctly.
  • Survivor table matches the source. src/labels/hold.ts:26 (required: true), :30 (comment?.body), :73-74 (empty-present guard + core.debug), and src/utils/duration.ts:13 (value === '' — on mutation, durationPart.exec('') yields null, consumed stays 0, result 0 either way) all read as the docs table claims.
  • Lockfile is clean. All 184 added resolved entries point at registry.npmjs.org; devDependencies only.
  • vitest.mutation.config.mjs excluding __tests__/bundle/** is sound: that harness executes the committed dist/index.js, which no src/ mutant reaches.

Scope matches the body: Refs #554 is appropriate since this lands only the mutation-check box of the tracker. CI is green. No findings — leaving the merge decision to a maintainer since the PR carries hold.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hold Indicates that a PR should not merge because someone has issued a /hold command. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants