Skip to content

sweep discards a validating result.json unread on any non-completed status; zero-hook-event runs pass run/sweep untouched #752

Description

@wsquared

Observed (0.10.0)

In sweep.py's triage handling:

if result.status != "completed":
    plan, errors = None, [session_failure_reason("triage", result)]
else:
    plan, errors = validate_triage(result.result_json, open_now)

When the session status is anything but completed, result.result_json is never read. After max_triage_attempts the run escalates with "triage output failed validation: ..." — blaming output that was never validated. The migration path has the same shape (validate_migration runs only on completed). #194's env_fault pause softens the transport-failure case only; a lost event channel is not classified as an env fault.

Incident

Run 20260816-113627-d0b2 (macOS, claude adapter). The hook relay had never been registered in that project, so every session's Stop event was lost and each session read as timeout with session_id: null. Both triage attempts wrote a result.json that passes validate_triage (strict and cache mode alike) — attempt 2 finished its actual work in 59 seconds — and both were discarded on session status alone. The escalation blamed the output; the output was fine. signals.py's own docstring predicts the failure mode: losing Stop events means every session stalls to session_timeout_min — "the loudest possible regression, delivered silently".

Proposals (either or both)

  1. Validate the result artifact on non-completed status too. If it validates, either use it — a timeout with a valid, complete artifact is a completed turn whose Stop event was lost — or at minimum attach "result artifact present and passes validation" to the escalation so the operator debugs the event channel rather than the agent. SessionResult.stop_seen (Session read-back adopts another story's spec: a review that produced nothing is scored done #261) is already the right discriminator: status == "timeout" and not stop_seen with a validating artifact is the lost-event-channel signature. Routing it like Dev session lost to an API/transport failure is charged as a story timeout, exhausting the attempt budget #194's env_fault pause (pause, don't charge attempts) would fit the existing shape.

  2. Make run/sweep enforce the zero-hook-events condition. cmd_validate fails on unregistered, missing, unreadable, or stale relays, but nothing stops a run whose sessions produce zero events: it burns session_timeout_min per session and escalates with the wrong blame. Hard-failing (or pausing) after the first session that ends with stop_seen == False and an empty run events dir would surface the miswiring at session one instead of N timeouts later.

Happy to attach the journal/state.json from the incident run.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions