Francesco Bertolaccini
555dbd6068
Adds the `diataxis` plugin: a skill and an eleven-agent dynamic workflow that
generates a four-quadrant documentation set, with the reference quadrant
produced by a real generator (Sphinx, Doxygen, rustdoc, TypeDoc) rather than
hand-written beside the code.
Also adds an eval suite under `evals/` — four fire cases and two should-NOT-fire
cases, run with `--ablation with-without` to measure uplift over an unaided
model. Full run at runs=3: mean Δ +0.26.
The suite establishes two things worth recording:
- When the skill fires, it delivers. `reference-generator-scaffolded` and
`doc-comments-in-source` pass on every fired run and on none of the nine
baseline runs; all four quadrants land in 9/9 fired runs.
- The trigger is unreliable. The parameterized form ("Document the parser
package and put the output in website/content") fired 0/3 despite being the
usage the README advertises, and the explicit form fired 2/3. Prompts that
state a need rather than name the framework fired 3/3.
Fire-case prompts explicitly authorize workflows and subagents, because eval
agents otherwise decline to use them and the plugin's only real path never
runs. Eval run output is gitignored; the cases are tracked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
diataxis
Generates a complete Diátaxis-structured documentation set for a codebase, using a dynamic workflow that orchestrates eleven agents across four phases.
What It Does
Diátaxis splits documentation into four kinds, each serving a different user need. Most projects blur them together — a README that is part tutorial, part reference, part rationale, and reliably wrong about all three. This plugin generates each kind separately, with the boundaries enforced.
Phase 1 Framework 1 agent clones diataxis-documentation-framework, distills the rules
Phase 2 Survey 5 agents parallel readers map the codebase
Phase 3 Author 4 agents one per quadrant, writing to disjoint paths
Phase 4 Assemble 1 agent index, cross-links, build wiring
Output lands in docs/ in the Diátaxis layout:
| Directory | Kind | Serves |
|---|---|---|
docs/tutorials/ |
Tutorial | A newcomer learning by doing |
docs/how-to/ |
How-to guide | A competent user with a task to finish |
docs/reference/ |
Reference | Someone who needs a fact, fast |
docs/explanation/ |
Explanation | Someone building a mental model |
Reference is generated from the code, not written beside it. The reference agent detects the language, scaffolds the appropriate generator (Doxygen, Sphinx + autodoc, rustdoc, godoc, TypeDoc, Javadoc, YARD, NatSpec), and fills in the missing doc comments in the source so the generated output is actually complete. Reference that lives anywhere but the code drifts from it within a release.
Why It Clones the Framework
The Diátaxis framework is published under CC-BY-SA 4.0. Rather than vendoring its text — which would carry a ShareAlike obligation into every repository this plugin runs against — the first agent clones evildmp/diataxis-documentation-framework at run time and distills the rules it needs. The plugin ships only its own guidance.
That first phase also acts as a correctness gate: it must return the resolved commit SHA and a count of source files it actually read. An agent that failed to clone and answered from memory fails the run instead of quietly producing a plausible-looking framework spec.
When to Use
- A project with a sprawling README and nothing else
- Documentation that exists but is organized by module rather than by reader need
- Before a public release, when
open-sourcingflags documentation as a gap
Not a good fit for a project with a mature, deliberately-structured docs site — the workflow adds to docs/, it does not reorganize what is already there.
Usage
/diataxis:document
Or describe the task and let the skill trigger: "write Diátaxis documentation for this repo".
The workflow takes optional arguments:
| Argument | Default | Meaning |
|---|---|---|
target |
. |
Path to the codebase to document |
docsDir |
docs |
Where the four quadrants are written |
Run /diataxis:document on src/parser with docs in website/content
Before You Run It
Commit or stash first. The reference agent edits source files to add doc comments, and workflow agents run in acceptEdits mode — their file writes are auto-approved. A clean worktree is what makes the entire run reviewable with git diff.
The run needs network access for the framework clone, and it spawns eleven agents, so it costs meaningfully more than a single-turn task. To gauge it, point target at one package first.
Installation
claude plugins:add trailofbits/skills/diataxis
If Dynamic Workflows Are Unavailable
Workflows can be turned off per-user (/config), per-organization (managed settings), or be absent on a non-Claude Code runtime. The skill detects this and runs the same four phases sequentially with the Agent tool instead. Both paths read the same agent briefs from skills/documenting-with-diataxis/references/agent-prompts.md, so they stay in step.
Components
| Path | Purpose |
|---|---|
workflows/document.js |
The dynamic workflow — runs as /diataxis:document |
skills/documenting-with-diataxis/SKILL.md |
Entry point, preflight, adaptation guidance, fallback |
references/diataxis-quadrants.md |
Boundary discipline: which kind a given page is |
references/reference-toolchains.md |
Language to doc generator, config, and build command |
references/agent-prompts.md |
The eleven agent briefs, shared by both execution paths |