Repository navigation
Description
Is your feature request related to a problem? Please describe.
The exact-selection contract needs a small ordinary scenario consumer. Other defaults could sample away selections, skip incompatible work, run unintended techniques or duplicate prompt sending as technique and baseline.
Part of #2744. Depends on #2757, including the completed scorer API. Keep not ready yet until promoted.
Describe the solution you'd like
Implement the smallest registered scenario/configuration adapter over resolved selections. Reuse Scenario, explicit DatasetAttackConfiguration groups, factories/builders, shared scorer policy, normal progress/history and saved-run plans. technique.execute deliberately handles one free-text objective without dataset inputs; reuse patterns, not repeated REST launches or an unrelated expansion of its contract.
Untouched settings produce exactly one prompt-sending execution per valid selected example: no additional converters, default cap, unintended technique set or duplicate baseline. Choose baseline/default behavior deliberately without changing other scenarios or adding a duplicate global factory as a workaround.
Preserve complete materialized conversations, per-turn media, roles/order/groups and provenance. Retain source objectives and require missing ones explicitly. Explain unsupported/ambiguous shapes and unresolved templates/configurations rather than flattening or generating them.
Reuse configured source-pinned scorers and shared default/expectation/condition validation. Validate the selected population and prerequisite targets before launch. True/false family alone is insufficient; raw refusal=true is not attack success. Invalid explicit choices must not silently fall back; no unscored mode.
Fail clearly for incompatible requested work rather than silently skipping examples/techniques or adapting selected content away. Reuse existing incompatible-technique policy/requirement checks, not a second service/DTO compatibility engine.
Acceptance criteria:
- More than five valid selections still yield N default prompt-sending executions, not five, N plus a baseline or another default technique set.
- Conversations/media, objectives/conditions and provenance survive execution and supported resume without reselecting/regenerating.
- Invalid groups, missing objectives, incompatible targets/techniques and scorer prerequisites produce actionable pre-execution errors; nothing silently disappears.
- Shared defaults, explicit non-default choices, stale references, composites/inverters and refusal-result meaning have focused coverage without duplicate policy.
- Preview/estimate/launch agree; existing scenarios retain their defaults/named-source behavior.
- Scenario/service tests use local fixtures and fake targets/scorers. Expose seams for existing Scanner UI, not a new runner.
Describe alternatives you've considered, if relevant
No GUI batch executor, generated scenario classes, global scorer redesign, provider loading, arbitrary config interpreter, memory migration or per-selection dataset. Optional additive converter UI #2758 is not required for the default consumer.
Additional context
Use current framework/scenario contracts, deliberate DatasetAttackConfiguration(seed_groups=..., max_total="all"), and saved run-plan/resume behavior. Current core already supplies condition validation and incompatible-technique policy; extend only demonstrated consumer gaps in the shared owner.
#2759 owns the existing-form integration and real-backend journey. Open #3526 is related form work, not a backend-adapter prerequisite. Scope narrowed November 4, 2026.
Activity
[-]FEAT: Support custom scenarios over selected dataset examples[/-][+]FEAT: Run a scenario over exactly selected dataset examples[/+]