conorbronsdon__avoid-ai-writing/detector

Detector engine

patterns.js is the executable expression of this skill's pattern rules — a zero-dependency, build-step-free detection engine that scores text for AI-writing tells. It runs identically in Node (>=18) and in the browser.

The skill's references/patterns.md is the human-readable catalog of rules; this engine is the deterministic, testable implementation of the regex-detectable subset, plus stylometric and AI-tool-fingerprint detectors that don't make sense as prose. See CATEGORIES.md for the rule ↔ category mapping that keeps the two in sync.

Run it

From npm

Install the published detector package in another project:

npm install avoid-ai-writing-detector

const AIDetector = require("avoid-ai-writing-detector");
const result = AIDetector.analyzeText("Your text here…");
console.log(result.score, result.label, result.issues.length);

From the command line

The published package exposes an avoid-ai-writing command that scores a file or stdin and prints the full result as JSON:

npx --package avoid-ai-writing-detector avoid-ai-writing draft.md
cat draft.md | npx --package avoid-ai-writing-detector avoid-ai-writing --context technical

As a CI or pre-commit gate

avoid-ai-writing-gate turns the existing detector into a pass/fail interface for automation. It intentionally gates on deterministic issues.length per file rather than the composite score:

avoid-ai-writing-gate --glob "**/*.md" --context technical
avoid-ai-writing-gate --threshold 0 docs/strict-policy.md
avoid-ai-writing-gate --threshold 2 docs/guide.md README.md
avoid-ai-writing-gate --json --glob "**/*.md"

Exit codes:

• 0: every scanned file is at or below the finding threshold;
• 1: at least one file exceeds the threshold;
• 2: usage, glob-expansion, file-read, UTF-8, or unscannable-input error (including documents above the detector's 10,000-word limit and unsegmented-script documents the engine declined to score).

The --json flag formats scan results as structured JSON on standard output with schemaVersion, per-file entries (path, findings, pass, types), and aggregates (pass, totalFindings, failedFiles).

The GitHub Action in action.yml exposes glob, threshold, context, and source-mode inputs, and outputs pass, total-findings, and failed-files. The CLI, Action, and shipped pre-commit hook default to 6 findings per file with technical context and rendered-markdown source mode.

That default is measured rather than guessed. On the current 376-document human control corpus under those exact settings, threshold 0 rejects 31.4% of human documents, while threshold 6 rejects 1.9% (7/376). A value of 6 is at or above the observed 95th-percentile finding count for every represented register; the single technical-blog document is an explicitly under-sampled slice. Use --threshold 0 when a repository deliberately wants a strict zero-findings policy. The corpus predates current model generations, so this threshold is a practical writing-quality default, not an AI-authorship accuracy claim.

This gate does not run validate.js: preservation checks compare two versions of a document, while CI/pre-commit detection inspects one snapshot. Run the preservation validator separately when an automated rewrite is part of the workflow.

From a local checkout

Use the repository directly when developing or validating detector changes:

npm test          # all suites; failures in one file still run the rest (no deps)
node scripts/run-tests.js detector/patterns.test.js   # one suite
node detector/patterns.test.js                        # same, direct

const AIDetector = require("./detector/patterns.js");
const result = AIDetector.analyzeText("Your text here…");
console.log(result.score, result.label, result.issues.length);

In the browser

Load patterns.js as a plain script — it self-registers as a global AIDetector (the module.exports block is guarded and only runs under CommonJS).

analyzeText(text, options?) → result

Field	Type	Meaning
score	0–100	0 = clean, 100 = heavy AI
label	string	scored: Clean (0) / Minimal AI signals (1–15) / Some AI patterns (16–35) / Moderate AI signals (36–60) / Strong AI signals (61–80) / Heavy AI patterns (81–100). Unscored: Empty / Too short / Unsupported script / Text too long
issues[]	{type, text, severity, …}	one entry per detected pattern; type keys map to CATEGORIES.md
stats	object	wordCount, per-tier counts, contextMode, sourceMode, masked-span counts, denseAIVocab, normalization flags, etc.
document_classification	string	HUMAN_ONLY / MIXED / AI_ONLY (shape mirrors GPTZero for swap-in), or UNSCORED on the early-exit paths
class_probabilities	{human, mixed, ai}	sums to exactly 1.0
confidence_category	low / medium / high
highlight_sentence_for_ai	region[]	sentence spans with source offsets + per-region score, for UI highlighting

The four unscored labels share one result shape: score 0, document_classification UNSCORED, an even class_probabilities split, and confidence_category low. Branch on that classification rather than on the score, since clean text also scores 0 and is labeled Clean. Unsupported script marks a document dominated by an unsegmented script (Chinese/Japanese: Han and kana characters, whose language has no inter-word spaces for countWords to split on) that was declined, not scored. An incidental place name or single Han character in otherwise English text does not qualify; Korean (Hangul) is space-separated and scores normally.

options.contextMode accepts general (default), technical, marketing, and personal. Technical mode suppresses flags that are legitimate in code-adjacent prose (e.g. Title Case headers and eight technical-legitimate terms: robust, comprehensive, seamless, ecosystem, leverage, facilitate, underpin, streamline); marketing and personal are accepted and reported in stats.contextMode, but currently score the same as general. Invalid modes fall back to general and set stats.contextModeFallback to the value you passed.

The skill's context profiles map to contextMode as follows:

Skill profile	Detector mode	What differs
linkedin	marketing	The skill applies the LinkedIn tolerance profile; detector marketing currently scores like general.
blog	general	The skill applies the default blog tolerance profile; detector uses baseline behavior.
technical-blog	technical	The skill applies technical-blog tolerances; detector enables technical-context suppressions.
investor-email	marketing	The skill applies stricter investor-email tolerances; detector marketing currently scores like general.
docs	technical	The skill applies docs tolerances; detector enables technical-context suppressions.
casual	personal	The skill applies casual tolerances; detector personal currently scores like general.

See references/patterns.md for the full context-profile definitions and tolerance matrix.

options.sourceMode accepts plain (default) or rendered-markdown. Rendered Markdown mode masks initial YAML frontmatter and HTML comments before pattern matching and document metrics run. Frontmatter may use LF, CRLF, or CR line endings and must begin with a YAML mapping entry after any leading blank or comment lines; this keeps ordinary prose between thematic breaks visible. Comment markers inside fenced or inline code remain visible code, while an actual unclosed comment is masked through end of file.

Masking preserves the input length and line endings so issue and sentence-highlight offsets still address the original source. The result reports sourceMode, sourceModeFallback, maskedFrontmatter, and maskedHtmlComments in stats. When an explicit invalid source mode falls back to plain, sourceModeFallback retains the requested value, including falsy values; without a fallback it is undefined.

Comment contents are fully excluded in rendered mode. Use plain mode or a source-hygiene linter when TODO placeholders inside comments should still be reported.

validate(original, rewritten, options?) → result

validate.js checks that a rewrite kept its hands off the things references/patterns.md says not to touch. Edit mode writes to files, so a violation there is silent and destructive.

const { validate, formatResult } = require("./detector/validate.js");
const result = validate(originalText, rewrittenText);
if (!result.ok) console.error(formatResult(result));

node detector/validate.js before.md after.md   # exits 1 on a preservation error

Errors (the rewrite altered content it had no business touching): fenced code modified or dropped, YAML frontmatter changed, blockquote reworded, table cell changed, inline code removed, URL or file path lost, heading count or nesting changed, and residual-grew when the rewrite introduces more flagged patterns than it removes.

Warnings (usually legitimate, occasionally a mistake): heading reworded, a figure from the original missing, more than 40% of the words dropped.

Two edits this skill documents as correct are carved out so the validator never fires on its own instructions: stripping AI tracking parameters from URLs (utm_source=chatgpt.com), and rewording a heading to fix Title Case or remove an emoji. Indented code blocks are counted but not enforced, since four-space indentation is also how markdown continues a list item.

Scoring our own docs

npm run self-scan          # table
npm run self-scan:check    # exits 1 if a document is over budget (runs in CI)

Results and the findings it surfaced are in ../PROOF.md.

Design notes

• FN-biased. False positives damage trust more than false negatives, so MIXED is wide and AI_ONLY requires multiple corroborating signals.
• Scoring is non-linear. Repeated hits of the same phrase are deduplicated; category weights live in the ISSUE_WEIGHTS table.
• Length gates. Under ~10 words → Too short (unscorable); over 10k words → Text too long.