feat: add an inspect-only writing audit - #261
Closed
mvanhorn wants to merge 1 commit into
Closed
Conversation
Add an inspect-only mode that reports findings without rewriting. The audit quotes each excerpt with its pattern number, explains why the match is contextual, and labels it clear, possible, or keep. A claim ledger lists rankings, qualifiers, numbers, names, dates, quotations, links, and simultaneity statements a rewrite must keep. The audit never scores the text or claims it is AI-generated. Mirror the mode in the README, route inspect intent in the skill description and OpenAI metadata, extend package validation to keep mode names and terms synchronized, and bump the version to 2.12.0. See blader#212, blader#229, blader#250. Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
Owner
|
Closing. A model that has read the skill and is asked to audit without rewriting already returns the pattern list from step 1 and stops. The PR fixes a four-part format and exactly three labels, and the validator then asserts those words in the prose. That freezes teaching text into a contract and grows the prompt for every user. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
Humanizer rewrites pasted text by default. Users who want to see which patterns fired, review a draft before allowing edits, or compare editorial choices must accept a rewrite first. This PR adds an inspect-only mode: a prompt-only audit that reports findings and changes nothing. It reuses the existing pattern taxonomy and false-positive rules, so there is no detector, score, or runtime service.
The mode triggers only on clear audit language such as "audit this text," "review this for AI patterns," or "show what you would change without rewriting." A plain rewrite or review request still goes through the normal rewrite path.
The audit returns four parts:
clear,possible, orkeep).The audit applies every false-positive rule, leaves quotations, examples, code, proper names, and deliberate voice alone, and never emits a detector score or a claim that the text is AI-generated.
Why
Changes
SKILL.md: addsInspect-only modeunder "How to return the result," defines the four-part audit schema and the three-label confidence vocabulary, and adds the claim ledger. The frontmatter description routes audit intent without capturing plain review requests. Version bumped to 2.12.0. The file stays under the 500-line limit (477 lines).README.md: mirrors the mode with one compact usage example and a sample findings row, and adds the 2.12.0 history entry.agents/openai.yaml: the short description and default prompt mention the audit path so inspect intent can route.scripts/validate-package.py: now asserts all four return-mode names exist inSKILL.mdand that the inspect-mode name and the three confidence labels stay synchronized betweenSKILL.mdandREADME.md, alongside the existing version and history checks..claude-plugin/plugin.json: version bumped to 2.12.0.Walkthrough
Ask
Audit this text for AI patterns without rewriting it: [text]. Humanizer runs the same pattern check it would run before a rewrite, then stops. You get the one-sentence assessment, the findings table with quoted excerpts and pattern numbers, the claim ledger, and the rewrite brief. Nothing is rewritten and no file changes. Since the skill is prompt-only Markdown, there is no runtime to capture; the mode is fully specified in theInspect-only modesection ofSKILL.md.Testing
python3 scripts/validate-package.pypasses at v2.12.0.keeplabel fromSKILL.md, or renaming the return mode each make the validator fail with a clear message.npx skills add . --listdiscovers the skill with the updated description.See #212, #229, #250.