[Manohar Paturi (ManoharPaturi)]

Description

Fixes #3627

_update_mcts_rewards divided the jailbreak count by the number of configured seed prompts while its num_queries parameter, the actual number of scores returned, sat unused except as a zero guard. Because the count comes from the flattened score_batch_async result, which may hold several scores per response, the reward could leave its intended range: three jailbreak scores over one prompt produced 3.0, inflating the MCTS exploitation term. The reward now divides by the scored query count, keeping it a true rate for any scorer.

Tests and Documentation

Added a regression test feeding a multi score batch shape that asserts the reward is exactly 1.0 when every score is a jailbreak and 0.25 for one in four. It fails on main, which returns 3.0 and 1.0 respectively for these inputs, and passes with the change. The full Fuzzer suite passes (49 tests).

No documentation change needed, so I did not run JupyText.

[Richard Lundeen (richlundeen)]

No material issues found. AI-assisted.

[Manohar Paturi (ManoharPaturi)]

Quick note on the red macOS job so it does not look like the change broke something: the failure is test_hierarchy_traversal_halts_immediately_on_budget_exhaustion in tests/unit/backend/test_scenario_hierarchy_parity.py, a wall clock assertion that expects the traversal under 0.05 seconds and measured 0.0539 on that runner. Nothing in this PR touches that path.

Main is flapping on the same test today. Run 41014283717 on main failed it an hour ago with 1.7288 seconds on ubuntu 3.13, while run 41012598348 passed the identical code at 06:11, and two earlier main runs yesterday evening failed it too. The test also passes locally on the PR branch every time I run it.

Since the approval and auto merge are already in place, one re-run of the failed job should be enough to land this whenever a runner is not contended.

Update: the re-run advice above was wrong. The same macOS leg failed three consecutive attempts, so re-running is a coin flip on this one, not a fix. The timing assertion itself is the flake; #3688 removes it in favor of the structural assertions that already prove the halt, and this PR should land once that merges and the branch updates.

[Manohar Paturi (ManoharPaturi)]

Filed #3671 with the main run evidence so the timing margin has a tracker.