Skip to content

feat: add an inspect-only writing audit - #261

Closed
mvanhorn wants to merge 1 commit into
blader:mainfrom
mvanhorn:cursor/inspect-only-writing-audit-3f7e
Closed

feat: add an inspect-only writing audit#261
mvanhorn wants to merge 1 commit into
blader:mainfrom
mvanhorn:cursor/inspect-only-writing-audit-3f7e

Conversation

@mvanhorn

@mvanhorn mvanhorn commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What this adds

Humanizer rewrites pasted text by default. Users who want to see which patterns fired, review a draft before allowing edits, or compare editorial choices must accept a rewrite first. This PR adds an inspect-only mode: a prompt-only audit that reports findings and changes nothing. It reuses the existing pattern taxonomy and false-positive rules, so there is no detector, score, or runtime service.

The mode triggers only on clear audit language such as "audit this text," "review this for AI patterns," or "show what you would change without rewriting." A plain rewrite or review request still goes through the normal rewrite path.

The audit returns four parts:

  1. A one-sentence assessment that does not classify authorship.
  2. A findings table: the exact excerpt, the matching pattern number, why the match is contextual rather than a banned word, and a confidence label (clear, possible, or keep).
  3. A claim ledger of the meaning-sensitive language a future rewrite must keep: rankings and superlatives, qualifiers, numbers, names, dates, quotations, links, and simultaneity statements.
  4. A short rewrite brief ordered by impact, with no rewritten sentences.

The audit applies every false-positive rule, leaves quotations, examples, code, proper names, and deliberate voice alone, and never emits a detector score or a claim that the text is AI-generated.

Why

Changes

  • SKILL.md: adds Inspect-only mode under "How to return the result," defines the four-part audit schema and the three-label confidence vocabulary, and adds the claim ledger. The frontmatter description routes audit intent without capturing plain review requests. Version bumped to 2.12.0. The file stays under the 500-line limit (477 lines).
  • README.md: mirrors the mode with one compact usage example and a sample findings row, and adds the 2.12.0 history entry.
  • agents/openai.yaml: the short description and default prompt mention the audit path so inspect intent can route.
  • scripts/validate-package.py: now asserts all four return-mode names exist in SKILL.md and that the inspect-mode name and the three confidence labels stay synchronized between SKILL.md and README.md, alongside the existing version and history checks.
  • .claude-plugin/plugin.json: version bumped to 2.12.0.

Walkthrough

Ask Audit this text for AI patterns without rewriting it: [text]. Humanizer runs the same pattern check it would run before a rewrite, then stops. You get the one-sentence assessment, the findings table with quoted excerpts and pattern numbers, the claim ledger, and the rewrite brief. Nothing is rewritten and no file changes. Since the skill is prompt-only Markdown, there is no runtime to capture; the mode is fully specified in the Inspect-only mode section of SKILL.md.

Testing

  • python3 scripts/validate-package.py passes at v2.12.0.
  • Negative tests: renaming the README section, removing the keep label from SKILL.md, or renaming the return mode each make the validator fail with a clear message.
  • npx skills add . --list discovers the skill with the updated description.

See #212, #229, #250.

Add an inspect-only mode that reports findings without rewriting.
The audit quotes each excerpt with its pattern number, explains why
the match is contextual, and labels it clear, possible, or keep.
A claim ledger lists rankings, qualifiers, numbers, names, dates,
quotations, links, and simultaneity statements a rewrite must keep.
The audit never scores the text or claims it is AI-generated.

Mirror the mode in the README, route inspect intent in the skill
description and OpenAI metadata, extend package validation to keep
mode names and terms synchronized, and bump the version to 2.12.0.

See blader#212, blader#229, blader#250.

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
@blader

blader commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Closing. A model that has read the skill and is asked to audit without rewriting already returns the pattern list from step 1 and stops. The PR fixes a four-part format and exactly three labels, and the validator then asserts those words in the prose. That freezes teaching text into a contract and grows the prompt for every user.

@blader blader closed this Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants