Skip to content

[quality] bundle fake answers unrouted requests with a plain 404; the harness should assert on unexpected traffic outside application fallbacks #568

Closed

Description

@hivecommons-hive

Finding

Child of #554 (P1 "Strengthen assertions and the test harness", second checkbox): __tests__/bundle/fakeGithub.ts records requests no route matched but answers them with a normal-looking 404, so each scenario is left to notice traffic it never modelled. Application fallbacks hide the miss: src/utils/mergeQueue.ts queueState swallows any error (including that 404) into a once-per-run warning and a direct merge, which is the #518 pattern.

Measured on main at e3119ad by instrumenting the fake's 404 arm during vitest run __tests__/bundle (86 files, 528 tests): 3,029 unmatched requests. ~2,800 are the configuration loader's probes and 175 the OWNERS tree probe, both intentional. The residue is 25 unmatched POST /graphql (tide's merge queue read, in 11 files that never route it) and ~20 membership/collaborator/OWNERS reads whose 404 is the point of the scenario. None of those is asserted as intentional anywhere; a forgotten route and a deliberate miss look identical.

Recommendation

  • Keep the unmatched requests apart in the fake and let a scenario declare the misses it relies on (allowUnmatched), with the configuration loader's probes always allowed.
  • Give the harness an assertion that fails a scenario on the rest (assertNoUnexpectedRequests), callable from afterEach ahead of reset.
  • Pin that contract with a harness-only test (plain fetches, no bundle run).
  • Adopt it in the OWNERS bundle suites whose merge queue read was unrouted (ownersParseArms.test.ts, ownersProbeArms.test.ts), answering that read through mergeQueueFallback as lgtmBindingArms/lgtmRemovalNoopArms already do.

Wider adoption across the other bundle files that leave POST /graphql unrouted (approveMembershipArms, cronLgtmFailureArms, ownersTreeLoadArms, ownersFilters, approveEvaluationArms, blunderbussArms, blunderbussDelay) is deliberately not part of this issue: the blunderbuss files are occupied by open hold PRs (#559, #561, #562, #567), and each remaining file is a one-line beforeEach/afterEach change that can follow once the harness is in.

Decision taken by hive: additive API on the fake, opt-in per file, loader probes allowed by default. Making reset() assert for every file would fail ~20 suites at once and is not backward compatible.

Priority

  • Impact: high (unmodelled traffic currently passes silently in bundle scenarios)
  • Effort: low

Filed by quality agent (hold-gated mode)


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5q9t | SHA: 6faa951

— hive: agent=quality backend=copilot model=claude-fable-5.1

Activity

  1. added
    testingCreated by Hive for agent-filed issue provenance
    agent/qualityCreated by Hive for agent-filed issue provenance
    on Oct 22, 2026
  2. github-actions commented on Oct 22, 2026

    @github-actions
    Contributor

    Please add a kind label with /kind feature, /kind failing-test or /kind cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityCreated by Hive for agent-filed issue provenanceneeds-kindtestingCreated by Hive for agent-filed issue provenance

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions