FEAT: Let each deployment configure its finding fields

doc/code/setup/finding_fields.md

@@ -0,0 +1,87 @@
+# Finding Fields
+
+A finding in CoPyRIT has a title, a severity and a set of fields. Each deployment chooses those fields. The finding dialog, the finding cards and the exports all follow them.
+
+## Default Fields
+
+Without any configuration, findings use these fields:
+
+- **Severity**: Critical, Important, Moderate, Low or Informational. People can also choose **Other** and type their own text. The dialog preselects Moderate.
+- **Harm-type**: optional. Its list is searchable, and it also takes **Other** text.
+- **Description**: optional multi-line text.
+
+## Defining Your Own Fields
+
+To use other fields, subclass `FindingsInitializer` and return a `FindingFields` from `get_finding_fields`. The `severity` slot is always a required choice field. The fields in `fields` follow it in the dialog.
+
+```python
+from pyrit.models import FindingField, FindingFields, FindingFieldType, FindingSeverityColor
+from pyrit.setup.initializers import FindingsInitializer
+
+
+class ContosoFindingsInitializer(FindingsInitializer):
+    """Finding fields for the Contoso red team."""
+
+    def get_finding_fields(self) -> FindingFields:
+        return FindingFields(
+            severity=FindingField(
+                key="severity",
+                label="Severity",
+                type=FindingFieldType.CHOICE,
+                options=[("sev1", "Sev 1"), ("sev2", "Sev 2"), ("sev3", "Sev 3")],
+                default="sev2",
+                colors={"sev1": FindingSeverityColor.RED, "sev2": FindingSeverityColor.ORANGE},
+            ),
+            fields=[
+                FindingField(
+                    key="area",
+                    label="Area",
+                    type=FindingFieldType.CHOICE,
+                    required=True,
+                    options={
+                        ("content", "Content safety"): [("hate", "Hate"), ("violence", "Violence")],
+                        ("security", "Security"): [("prompt_injection", "Prompt injection")],
+                    },
+                    levels=[("subarea", "Subarea")],
+                ),
+                FindingField(key="repro_steps", label="Repro steps", type=FindingFieldType.MULTILINE),
+                FindingField(key="contoso_ticket", label="Contoso ticket", type=FindingFieldType.TEXT, internal=True),
+            ],
+        )
+```
+
+A few things to note in this example:
+
+- Options are `(key, label)` pairs. The finding stores the `key`, and people see the `label`. Field and level keys use lowercase letters, digits and underscores. Option keys can be any text that isn't blank and has no surrounding whitespace.
+- `Area` is a cascade. Each of its options maps to the options of a second dropdown. `levels` names that dropdown, and its value is stored under `subarea`.
+- `colors` sets the badge color of a severity option.
+- `internal=True` keeps `Contoso ticket` out of the Word and Markdown exports. The dashboard still shows it.
+
+`FindingField` also takes `hint`, `section` and `allow_other`. Its docstring describes each setting.
+
+Save the script as `contoso_findings.py` in your custom initializers source. The file name is the initializer's name. Then turn it on in `.pyrit_conf`:
+
+```yaml
+allow_custom_initializers: true
+custom_initializers_source: /srv/contoso/pyrit_initializers
+initializers:
+  - name: target
+  - name: technique
+  - name: contoso_findings
+```
+
+With `allow_custom_initializers: true`, the backend runs every script in that source. Only trusted administrators should be able to write to it. See [the configuration file guide](../../getting_started/pyrit_conf.md) for details.
+
+## Rules
+
+- Use one `FindingsInitializer` per deployment. If the configuration lists two, or the fields are invalid, setup fails when the backend starts. The backend still runs, but it blocks runtime operations, including findings, until a process restart. Configuration repair stays available. A live apply reports invalid configuration instead, and the running fields don't change.
+- Don't make `get_finding_fields` depend on environment variables. A live apply checks the fields before it applies the new environment.
+- Change a field's `label` to rename what people see.
+- A changed `key` makes a new field. Values saved under the old key show under **Retired fields**, where they are read-only. Edits keep them. **Remove** deletes one when you save.
+- Other text that matches an option's key or label is rejected, ignoring case and surrounding spaces. The message names the option to choose instead.
+- On the Operations list, severities that match no option share one **Other** count.
+- Exports put multi-line fields in the Finding column, and choice and text fields in the Assessment column. They leave out internal fields and retired values. The export dialog warns when selected findings have retired values.
+
+## Upgrading
+
+One database migration moves findings saved before finding fields existed. It stores Other severity text as the `severity` itself, and moves the harm type and description into field values. Finding IDs and evidence don't change. Downgrading that migration refuses while any finding exists.

doc/gui/0_gui.md

@@ -213,8 +213,10 @@ This is a cross-result report, not the progress dashboard for one scanner run.
 ## Operations
 
 Operations organize human findings for an engagement. Create an operation,
-record findings with severity and description, and attach saved attacks as
-evidence. A model score does not create a finding automatically.
+record findings, and attach saved attacks as evidence. Findings use the
+deployment's [finding fields](../code/setup/finding_fields.md), which by default
+are severity, harm type, and description. A model score does not create a
+finding automatically.
 
 Select an operation in the labels bar to associate new chats and scans with it.
 The operation page brings its findings, linked evidence, and execution history

doc/myst.yml

@@ -192,6 +192,7 @@ project:
             - file: code/setup/1_configuration.ipynb
             - file: code/setup/2_resiliency.ipynb
             - file: code/setup/default_values.md
+            - file: code/setup/finding_fields.md
             - file: code/setup/pyrit_initializer.ipynb
         - file: code/scenarios/0_scenarios.ipynb
           children:

frontend/e2e/operations.spec.ts

@@ -3,7 +3,7 @@ import { readFileSync } from 'node:fs'
 import { test, expect } from '@playwright/test'
 
 import type {
-  CreateAttackResponse, Finding, FindingEvidenceAttachResponse, FindingExportFormat, Operation,
+  CreateAttackResponse, Finding, FindingEvidenceAttachResponse, FindingExportFormat, FindingFields, Operation,
 } from '../src/types'
 import { waitForAppReady } from './_runtime'
 
@@ -21,7 +21,7 @@ for (const width of [1280, 390]) {
     const severity = `Custom severity ${'classification'.repeat(16)}`
     const harmType = `Custom harm ${'unbroken'.repeat(25)}`
     const findingResponse = await request.post(`/api/operations/${operation.id}/findings`, {
-      headers, data: { title, severity: 'other', severity_other: severity, harm_type: 'Other', harm_type_other: harmType },
+      headers, data: { title, severity, field_values: { harm_type: harmType } },
     })
     expect(findingResponse.status()).toBe(201)
 
@@ -65,10 +65,7 @@ for (const width of [1280, 390]) {
     )).toBeTruthy()
     const label = dialog.locator('label').filter({ hasText: title })
     expect(await label.evaluate(element => element.scrollWidth <= element.clientWidth)).toBeTruthy()
-    const pickerBadgeBox = await pickerBadge.boundingBox()
-    const harmBox = await dialog.getByText(`Harm-type: ${harmType}`, { exact: true }).boundingBox()
-    if (!pickerBadgeBox || !harmBox) throw new Error('Expected visible picker classifications.')
-    expect(pickerBadgeBox.y + pickerBadgeBox.height).toBeLessThanOrEqual(harmBox.y)
+    await expect(dialog.getByText(`Harm-type: ${harmType}`, { exact: true })).toHaveCount(0)
     expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBeTruthy()
     await radio.focus()
     await page.keyboard.press('Space')
@@ -116,7 +113,8 @@ test('creates a unique operation and records findings within it @seeded', async
   dialog = page.getByRole('dialog')
   await expect(dialog.getByRole('combobox', { name: 'Operation' })).toHaveCount(0)
   await dialog.getByRole('textbox', { name: 'Title' }).fill(title)
-  await dialog.getByRole('combobox', { name: 'Severity' }).selectOption('informational')
+  await dialog.getByRole('combobox', { name: 'Severity' }).click()
+  await page.getByRole('option', { name: 'Informational', exact: true }).click()
   await dialog.getByRole('textbox', { name: 'Description' }).fill(description)
   const savedResponse = page.waitForResponse(response =>
     response.url().endsWith(`/api/operations/${operation.id}/findings`) && response.request().method() === 'POST')
@@ -145,7 +143,7 @@ test('creates a unique operation and records findings within it @seeded', async
   const headers = { 'PyRIT-Compatibility-ID': version.compatibility_id }
   for (let index = 0; index < 21; index++) {
     const response = await request.post(`/api/operations/${operation.id}/findings`, {
-      headers, data: { title: `Assessment ${index}`, description: '', severity: index === 20 ? 'critical' : 'low' },
+      headers, data: { title: `Assessment ${index}`, severity: index === 20 ? 'critical' : 'low' },
     })
 
     expect(response.status()).toBe(201)
@@ -189,7 +187,7 @@ test('renames an operation and deletes it only once it is empty @seeded', async
     headers, data: { name: `Typo operation ${stamp}` },
   })).json()
   const finding: Finding = await (await request.post(`/api/operations/${operation.id}/findings`, {
-    headers, data: { title: 'Blocks deletion', description: '', severity: 'low' },
+    headers, data: { title: 'Blocks deletion', severity: 'low' },
   })).json()
 
   await page.goto(`/operations/${operation.id}`)
@@ -305,7 +303,7 @@ test('selects saved operations inline and revisits ID-scoped attack history @see
     expect(other.ok()).toBeTruthy()
   }
   const finding = await request.post(`/api/operations/${operation.id}/findings`, {
-    headers, data: { title: 'Independent human finding', description: '', severity: 'low' },
+    headers, data: { title: 'Independent human finding', severity: 'low' },
   })
   expect(finding.status()).toBe(201)
   if (!sourceAttack) throw new Error('Expected a saved attack for the evidence test.')
@@ -372,7 +370,7 @@ test('exports selected findings as Word and Markdown @seeded', async ({ page, re
   // Saved out of severity order, so the document's order can only come from severity.
   const seeds = [
     { title: 'Export low', severity: 'low' },
-    { title: 'Export critical', severity: 'critical', description },
+    { title: 'Export critical', severity: 'critical', field_values: { description } },
     { title: longTitle, severity: 'informational' },
     ...Array.from({ length: 17 }, (_, index) => ({
       title: `Export moderate ${String(index).padStart(2, '0')}`, severity: 'moderate',
@@ -511,3 +509,63 @@ test('exports selected findings as Word and Markdown @seeded', async ({ page, re
   await page.goto(link)
   await expect(page.getByText('Saved offline evidence.', { exact: true })).toBeVisible()
 })
+
+test('keeps the page usable after removing a retired value and cancelling the edit @seeded', async ({
+  page, request,
+}) => {
+  const version = await (await request.get('/api/version')).json()
+  const headers = { 'PyRIT-Compatibility-ID': version.compatibility_id }
+  const fieldsResponse = await request.get('/api/operations/finding-fields', { headers })
+  expect(fieldsResponse.ok()).toBeTruthy()
+  const fields: FindingFields = await fieldsResponse.json()
+  const operationResponse = await request.post('/api/operations', {
+    headers, data: { name: `Retired values ${Date.now()}` },
+  })
+  expect(operationResponse.status()).toBe(201)
+  const operation: Operation = await operationResponse.json()
+  const title = 'Contoso retired notes'
+  const notes = 'Recorded before Contoso changed its finding fields.'
+  const findingResponse = await request.post(`/api/operations/${operation.id}/findings`, {
+    headers, data: { title, severity: 'low', field_values: { description: notes } },
+  })
+  expect(findingResponse.status()).toBe(201)
+  // The seeded backend accepts only known keys, so the browser gets the fields without Description instead,
+  // which makes the saved description a retired value.
+  await page.route('**/api/operations/finding-fields', async route => {
+    await route.fulfill({ json: { ...fields, fields: fields.fields.filter(field => field.key !== 'description') } })
+  })
+
+  await page.goto(`/operations/${operation.id}`)
+  await waitForAppReady(page)
+  const edit = page.getByRole('button', { name: `Edit finding: ${title}`, exact: true })
+  const dialog = page.getByRole('dialog', { name: 'Edit finding' })
+  const retired = dialog.getByRole('group', { name: 'Retired fields · read-only' })
+  const removeButton = retired.getByRole('button', { name: 'Remove retired value description', exact: true })
+  await edit.click()
+  await expect(retired.getByText(notes, { exact: true })).toBeVisible()
+  const confirmation = page.getByRole('dialog', { name: 'Remove description?' })
+  await removeButton.click()
+  await confirmation.getByRole('button', { name: 'Keep it', exact: true }).click()
+  await expect(confirmation).toHaveCount(0)
+  await expect(removeButton).toBeFocused()
+  await removeButton.click()
+  await expect(confirmation).toBeVisible()
+  await page.keyboard.press('Escape')
+  await expect(confirmation).toHaveCount(0)
+  await expect(dialog).toBeVisible()
+  await expect(removeButton).toBeFocused()
+
+  await removeButton.click()
+  await confirmation.getByRole('button', { name: 'Remove', exact: true }).click()
+  await expect(confirmation).toHaveCount(0)
+  await expect(dialog.getByRole('button', { name: 'Undo', exact: true })).toBeFocused()
+  await expect(dialog.getByText('1 retired value is removed when you save.', { exact: true })).toBeVisible()
+  await expect(retired).toHaveCount(0)
+  await dialog.getByRole('button', { name: 'Cancel', exact: true }).click()
+  await expect(page.getByRole('dialog')).toHaveCount(0)
+
+  await edit.click()
+  await expect(dialog).toBeVisible()
+  await expect(retired.getByText(notes, { exact: true })).toBeVisible()
+  await expect(removeButton).toBeVisible()
+})