bug: reconciliation can fail a Docker sandbox during restart #4981

User Story

As the OpenShell release captain, I need sandbox stop/start and main's Docker E2E qualification to be reliable so I can seed the v0.1.7 prerelease train from a passing Release Dev commit. This failure directly blocked that release workflow.

Problem Statement

A reconciliation sweep can apply an observation of the stopped Docker container after the gateway has committed a new Starting transition but before the driver finishes restarting it. The gateway records ContainerExited and makes the sandbox Error, even though the replacement supervisor subsequently authenticates successfully. The SSH host-identity E2E then fails at sandbox start, before checking the restarted host key.

Evidence: Release Dev run 40562846292 at 2c31ac7, job https://github.com/NVIDIA/OpenShell/actions/runs/40562846292/job/122402557846.

Impact / Why This Matters

A valid restart can fail intermittently and leave a healthy replacement marked Error. This blocks main qualification and starting the next prerelease train. Rerunning may avoid the timing window but does not fix lifecycle correctness.

Acceptance Criteria

• Reconciliation cannot apply a previous-run container exit over an active explicit start.
• Reconciliation still records genuine exits after the lifecycle operation settles.
• A deterministic regression test covers reconciliation while the driver start is blocked, including unrelated sandboxes remaining reconcilable.
• The SSH host-identity Docker E2E passes through stop/start without relaxing its assertions.

Reproduction Steps

1. Run an OpenShell gateway with the Docker compute driver and normal background reconciliation.
2. Create a sandbox and establish pinned SSH access.
3. Stop it and wait for Stopped.
4. Start it while a reconciliation sweep observes the retained stopped container before the driver completes startup. This interleaving is timing-dependent.
5. Observe ContainerExited from sandbox start even though the replacement supervisor connects.

The existing ssh_host_identity_survives_restarts_and_changes_after_recreation test hit this interleaving in Release Dev. A controlled-driver regression test should reproduce it deterministically without Docker timing dependence.

Environment

• OpenShell main: 2c31ac7
• Runtime: Docker, Release Dev's rust-docker E2E lane
• Gateway and tests use the same source commit.

Logs

UTC timestamps for the same sandbox, shortened:

19:11:53.371 StopSandbox completed successfully
19:11:53.585 reconcile: Starting -> Error, ContainerExited
19:11:54.071 replacement supervisor session accepted, auth_epoch=2
19:11:54.211 start response persistence: resource version conflict
19:11:55.218 test failed: ContainerExited while waiting for Ready

Investigation

The driver-watch path waits on the per-sandbox lifecycle gate and refreshes its observation for ambiguous lifecycle snapshots. reconcile_snapshot_sandbox instead checks resource version/timestamps and calls apply_sandbox_update_locked directly. Those checks do not exclude a start that began before the sweep. Investigate fencing or deferring sweep observations during active lifecycle work while retaining per-sandbox concurrency and genuine failure reporting.