feat: add -kb to make page type classification opt-in

[dogancanbakir]

Closes #2543

-json/-csv initialized the dit classifier, downloading a ~92MB model with no way to opt out. Classification is now opt-in via -kb, implied by -fpt/-fep.

Supersedes #2544 (thanks @jatinder14).

Summary by CodeRabbit

• New Features

  • Added knowledge base classification through the -knowledge-base and -kb command-line options.
  • Knowledge base classification is automatically enabled when page-type or legacy error-page filters are used.

• Bug Fixes

  • Improved classifier initialization across supported classification scenarios.
  • Classification setup failures are now reported clearly instead of being ignored.

• Documentation

  • Updated usage documentation with the new knowledge base option.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 253482d1-fc97-4d07-87be-30c4cab46d09

📥 Commits

Reviewing files that changed from the base of the PR and between 9bcc668 and 1be2364.

📒 Files selected for processing (1)

• runner/runner.go

Included review availability: 4 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

---

Walkthrough

The change adds knowledge-base classification configuration, derives classification from page-type filters, updates output settings, and changes classifier initialization errors to returned wrapped errors.

Changes

Classification control

Layer / File(s)	Summary
Classification options and derived state runner/options.go, README.md	Options now includes KnowledgeBase. The -knowledge-base and -kb flags set this field. Page-type filters and deprecated error-page filtering enable classification. The option is documented.
Conditional runner initialization runner/runner.go	The runner initializes dit when classification is enabled. Classifier initialization failures return as wrapped errors.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 1be23

The change makes page-type classification opt-in, reducing unnecessary model downloads while preserving classification when explicitly requested; no actionable merge-blocking risk remains.

Sequence Diagram(s)

sequenceDiagram
  participant RunnerOptions
  participant Runner
  participant dit
  RunnerOptions->>Runner: Enable classification from flags or filters
  Runner->>dit: Create classifier with dit.New()
  dit-->>Runner: Return classifier or initialization error
  Runner-->>RunnerOptions: Continue or return wrapped error

Loading

Suggested reviewers: ayanrajpoot10

Poem

A rabbit adds a flag today,
Knowledge-base paths now guide the way.
Filters wake the classifier bright,
Wrapped errors keep the startup right.
Hop, hop—the runner is prepared! 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the primary change: adding the -kb flag to make page-type classification opt-in.
Linked Issues check	✅ Passed	The PR meets [#2543] by making classification opt-in, avoiding unnecessary model downloads, and preserving classification for relevant filters.
Out of Scope Changes check	✅ Passed	All changes support the linked issue and PR objective; no unrelated code or documentation changes are present.
Docstring Coverage	✅ Passed	Docstring check was indeterminate for this PR — some files could not be analyzed in time. Not blocking.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat-kb-opt-in

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@runner/classifier.go`:
- Around line 19-21: Protect the temporary default logger replacement in
Runner.New with a package-level mutex, locking before slog.Default is saved and
slog.SetDefault installs gologgerSlogHandler, and unlocking only after the
deferred restoration completes. Ensure the mutex covers the entire save,
replacement, and restore sequence so overlapping Runner.New calls cannot leave
the temporary handler installed.

In `@runner/runner.go`:
- Around line 434-442: Update the newDitClassifier error handling within the
classificationEnabled path to return the wrapped initialization error
unconditionally, removing the hasPageTypeFilter distinction and non-fatal
logging path. Ensure every enabled-classification failure prevents returning a
Runner with a nil classifier.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3be27126-e8e0-40d8-82b4-83d33926eb23

📥 Commits

Reviewing files that changed from the base of the PR and between 8114098 and cb58e92.

📒 Files selected for processing (5)

• README.md
• runner/classifier.go
• runner/options.go
• runner/runner.go
• runner/runner_test.go

[jalilm]

@dogancanbakir the failing test was fixed in #2555, can you rebase and merge this change? thanks