fix(sleep): separate optimizer feedback from judge evidence

[diqierjia]

Summary

Related to #154.

This change addresses one concrete, reproducible failure mode: raw judge evidence can leak regexes and checker implementation details into SkillOpt-Sleep optimizer prompts. The optimizer may then learn verifier syntax instead of user-visible behavior.

Scope

• Add an optional ReplayResult.optimizer_feedback channel for learning-safe semantic feedback.
• Keep fail_reason and judge_rationale unchanged as raw audit/debug evidence.
• Route ordinary reflection, contrastive reflection, and slow update through semantic feedback only.
• Prefer a check's natural-language description; if it is absent, regex checks use a conservative non-pattern fallback.
• Fail closed when a regex is copied into its description or when legacy non-rule evidence cannot be safely projected.
• Preserve check descriptions emitted by the miner.
• Protect the contributed OpenClaw reflection path from bypassing the shared safety boundary.

Data flow

Raw judge rationale remains available in replay results, evidence logs, diagnostics, and reports for auditability. Optimizer prompts receive only optimizer_feedback (or a recomputed safe projection for legacy rule results). Rule scoring keeps the existing backward-compatible score_rule_judge() return shape; the new helper carries the additional semantic feedback.

Integration receipt

tests/test_optimizer_feedback_flow.py is a deterministic, no-network end-to-end regression using the real sleep-cycle orchestration:

miner description -> replay -> optimizer prompt -> proposal/audit artifacts

Two synthetic rule tasks provide train/validation separation. The baseline produces no required route and scores 0.0; the optimizer receives only the semantic description, proposes a general routing rule, and the candidate scores 1.0. Assertions verify that the private regex remains available in miner/replay evidence and diagnostics, while it is absent from the optimizer prompt, returned edit, report, and staged proposal.

This demonstrates preserved useful signal for this synthetic rule-judge path only. It does not establish unchanged optimization quality for every judge type or real workload.

Backward compatibility

optimizer_feedback is optional and defaults to an empty string. Older or manually deserialized ReplayResult objects without that attribute remain usable: rule tasks recompute feedback from the task judge, while exact/rubric and other legacy non-rule results use a generic message rather than exposing raw rationale. This fallback intentionally loses potentially useful diagnostic detail; richer non-rule learning feedback requires a future typed, explicitly safe feedback API.

Trust boundary

A check description is a targeted semantic projection expected to come from a trusted operator or miner. It is not a universal sanitizer for arbitrary untrusted prose, and this change is not a solution to the broader alignment problem. A description that literally copies its regex is rejected, but mined tasks, raw evidence, and staged proposals still require review.

Validation

• Focused regression suite: 221 passed, 3 skipped, 6 subtests passed.
• Full local suite: 1445 passed, 74 skipped, 353 subtests passed, 2 failed. Both failures are Windows symlink-privilege cases blocked by WinError 1314:
  • TestHarnessEvidence.test_pytest_after_edit_fails_closed_on_broken_source_symlink
  • TestCLIFailClosed.test_symlinked_candidate_refused
• python -m compileall -q skillopt skillopt_sleep plugins scripts tests: passed.
• mkdocs build --strict: passed.
• ruff check tests/test_optimizer_feedback_flow.py plugins/openclaw/skillopt_sleep_openclaw.py: passed.
• git diff --check: passed.

Non-goals

This is intentionally not a solution for the full intent-level mining or proxy-alignment problem described in #154. It is a narrow containment fix for judge-evidence leakage while preserving existing scoring and audit behavior.

[diqierjia]

@microsoft-github-policy-service agree

[Yifan Yang (Yif-Yang)]

Reviewed draft 53e3facd5a56 as an isolated test merge onto main at 79124b37e9a6. The Linux/Python 3.11 suite passes: 1511 passed, 9 skipped. I did not reproduce the Windows symlink-permission failures reported in the description on this host.

The separation has clear value: raw judge evidence remains auditable, while ordinary reflection, contrastive reflection, slow update, and the contributed OpenClaw path use the semantic projection. The compatibility wrapper and legacy-result regressions are also helpful.

Before marking this ready, please add a short reviewer receipt or an end-to-end integration regression tracing a mined description through replay, optimizer input, and the resulting proposal/audit artifacts. The existing focused tests cover the individual paths; the remaining acceptance question is whether the complete workflow retains useful learning signal without feeding verifier syntax back into the proposal.

In particular, include a small before/after example for rule feedback and an exact/rubric or legacy non-rule case. The generic non-rule fallback is intentionally conservative, but it also removes information that was previously available to reflection; please make that trade-off and any observed effect on improvement explicit rather than equating “no regex in a prompt” with “optimization quality preserved.” Use only synthetic/public examples, not private transcripts.

Please also document the trust boundary of operator/miner-provided descriptions: this is a targeted projection of known judge evidence, not a universal sanitizer for arbitrary untrusted prose or a solution to the entire alignment problem. Once the remaining draft work is complete, mark it ready and obtain green official CI on the final unchanged head. I am not treating the current draft as merge-ready, nor claiming a newly reproduced runtime regression here.

[diqierjia]

Addressed the remaining draft feedback on final head 250cb667ed1209cf6e827456978d27598bfbfee9:

1. Added tests/test_optimizer_feedback_flow.py, a deterministic no-network end-to-end regression covering miner description -> replay -> optimizer prompt -> proposal/audit artifacts through the real sleep-cycle orchestration.
2. Added a rule-judge before/after example to docs/sleep/README.md, including the synthetic 0.0 -> 1.0 result and its limitations.
3. Added an exact/rubric/legacy non-rule example and explicitly documented that the generic fail-closed fallback loses potentially useful learning signal.
4. Documented the trust boundary: descriptions are targeted trusted semantic projections, not a universal sanitizer for arbitrary untrusted prose.
5. Rebased onto current main, reran the focused/full local validation and strict docs build, and am marking the PR ready. I will treat official CI on this unchanged head as the remaining acceptance gate.

Local validation: focused 221 passed, 3 skipped; full 1445 passed, 74 skipped, 2 failed, with both failures limited to Windows WinError 1314 symlink privilege cases. mkdocs build --strict, compileall, focused Ruff, and git diff --check pass.

[diqierjia]

Official CI status for final unchanged head 250cb667ed1209cf6e827456978d27598bfbfee9: workflow run 34021182061 is currently action_required with zero jobs, so no test has failed or run yet. It requires upstream approval for the fork-originated workflow. Once approved, this same head should be the one evaluated for the requested green CI gate.