trailofbits__skills/AGENTS.md

Contributing Skills

Resources

Official Anthropic documentation (always check these first):

• Claude Code Plugins
• Agent Skills
• Best Practices
• Skill Authoring Best Practices — progressive disclosure, degrees of freedom, workflow checklists
• The Complete Guide to Building Skills (text) — evaluation-driven development, iterative testing

Reference skills - learn by example at different complexity levels:

Complexity	Skill	What It Demonstrates
Basic	git-cleanup	Single self-contained SKILL.md, allowed-tools scoping
Intermediate	constant-time-analysis	Python package, references/, language-specific docs
Advanced	culture-index	Scripts, workflows/, templates/, PDF extraction, multiple entry points

When in doubt, copy one of these and adapt it.

Deep dives on skill authoring:

• Claude Skills Deep Dive - Comprehensive analysis of skill architecture

Example plugins worth studying:

• superpowers - Advanced workflow patterns, TDD enforcement, multi-skill orchestration
• compound-engineering-plugin - Production plugin structure
• getsentry/skills — Production Sentry skills; security-review is a standout routing + progressive disclosure example

For Claude: Use the claude-code-guide subagent for questions about official Claude Code behaviour that you cannot answer from this repository — it has access to the official documentation. For anything answerable by reading the files here, read them; delegating a lookup you could do directly costs a round trip and buys nothing.

Technical Reference

Codex Compatibility

This repository uses Claude plugin marketplace metadata as the canonical source for both Claude Code and Codex. Codex supports .claude-plugin/marketplace.json and plugins/<name>/.claude-plugin/plugin.json directly, so do not add duplicate Codex-only sidecar metadata.

Rules:

• Do not add .agents/plugins/marketplace.json, .codex/, .opencode/, or plugins/<name>/.codex-plugin/. The validator enforces this — sidecars drift out of sync with the canonical metadata, which is why the last set was removed in #173.

• Keep plugin components at the plugin root using Codex-compatible default paths: skills/, hooks/hooks.json, .mcp.json, and .app.json.

• If a plugin needs MCP configuration, put it in .mcp.json at the plugin root rather than embedding an object in .claude-plugin/plugin.json.

• Both loadability checks run in CI, not in make check — they need the Claude Code and Codex CLIs installed, which is not a reasonable local prerequisite. Run them by hand if you are changing plugin metadata:

  python3 .github/scripts/check_claude_loadability.py
  python3 .github/scripts/check_codex_loadability.py
  

  If one fails, update the canonical Claude marketplace or the plugin root components so Codex can load them through Claude marketplace compatibility — do not add a sidecar to work around it.

Plugin Structure

plugins/
  <plugin-name>/
    .claude-plugin/
      plugin.json         # Plugin metadata (name, version, description, author)
    commands/             # Optional: slash commands
    agents/               # Optional: autonomous agents
    workflows/            # Optional: dynamic workflows (*.js), ships as /<plugin>:<workflow>
    evals/                # Optional: `claude plugin eval` cases + graders
    skills/               # Optional: knowledge/guidance
      <skill-name>/
        SKILL.md          # Entry point with frontmatter
        references/       # Optional: detailed docs
        workflows/        # Optional: step-by-step guides (prose, not scripts)
        scripts/          # Optional: utility scripts
    hooks/                # Optional: event hooks
    tests/                # Optional: run_*.sh suites (CI runs these — keep them free)
    README.md             # Plugin documentation

Important: Component directories (skills/, commands/, agents/, hooks/) must be at the plugin root, NOT inside .claude-plugin/. Only plugin.json belongs in .claude-plugin/.

Two different workflows/. A dynamic workflow is a JavaScript file at the plugin root under workflows/; it exports a meta object, orchestrates subagents in code, and ships through the marketplace as /<plugin-name>:<workflow-name> (from meta.name, not the filename). The older skills/<skill>/workflows/ holds prose step-by-step guides. If a SKILL.md has "Phase 1", "for each finding", or "repeat until" sections, the plan belongs in a script at the plugin root, not in prose. See plugins/variant-analysis/workflows/ for a worked example, and note that ${CLAUDE_PLUGIN_ROOT} is not available inside a workflow script — it is not a hook, MCP, or LSP subprocess.

Frontmatter

Skills and commands declare tools with allowed-tools, space-delimited:

---
name: skill-name              # kebab-case, max 64 chars
description: "Third-person description of what it does and when to use it"
allowed-tools: Read Grep      # Optional: restrict to needed tools only
---

Agent files under agents/ use a different key — tools, as a YAML list:

---
name: agent-name
description: "What this agent does and when the coordinator should dispatch it"
tools:
  - Read
  - Grep
---

The keys are inverted between the two file types, and the loader silently ignores the wrong one — the frontmatter still parses, the restriction simply does not apply and the agent inherits everything. The validator checks this.

Naming Conventions

• kebab-case: constant-time-analysis, not constantTimeAnalysis
• Gerund form preferred: analyzing-contracts, processing-pdfs (not contract-analyzer, pdf-processor)
• Avoid vague names: helper, utils, tools, misc
• Avoid reserved words: anthropic, claude

Path Handling

• Use {baseDir} for paths, never hardcode absolute paths
• Use forward slashes (/) even on Windows

Python Scripts

When skills include Python scripts with dependencies:

1. Use PEP 723 inline metadata - Declare dependencies in the script header:

   # /// script
   # requires-python = ">=3.11"
   # dependencies = ["requests>=2.28", "pydantic>=2.0"]
   # ///
   

2. Use uv run - Enables automatic dependency resolution:

   uv run {baseDir}/scripts/process.py input.pdf
   

3. Include pyproject.toml - Keep in scripts/ for development tooling (ruff, etc.)

4. Document system dependencies - List non-Python deps (poppler, tesseract) in workflows with platform-specific install commands

Hooks

PreToolUse hooks run on every Bash command—performance is critical:

• Prefer shell + jq over Python—interpreter startup (Python + tree-sitter) adds noticeable latency
• Fast-fail early - exit 0 immediately for non-matching commands so most invocations are instant
• Favor regex over AST parsing - accept rare false positives if performance gain is significant and Claude can rephrase
• Anticipate false positive patterns - diagnostic commands (which python), search tools (grep python), and filenames (cat python.txt) shouldn't trigger interception
• Document tradeoffs in PR descriptions so reviewers understand deliberate design choices

Quality Standards

These are Trail of Bits house standards on top of Anthropic's requirements.

Description Quality

Your skill competes with 100+ others. The description must trigger correctly.

• Third-person voice: "Analyzes X" not "I help with X"
• Include triggers: "Use when auditing Solidity" not just "Smart contract tool"
• Be specific: "Detects reentrancy vulnerabilities" not "Helps with security"

Value-Add

Skills should provide guidance Claude doesn't already have, not duplicate reference material.

• Behavioral guidance over reference dumps - Don't paste entire specs; teach when and how to look things up
• Explain WHY, not just WHAT - Include trade-offs, decision criteria, judgment calls
• Document anti-patterns WITH explanations - Say why something is wrong, not just that it's wrong

Example: The DWARF skill doesn't include the full DWARF spec. It teaches Claude how to use dwarfdump, readelf, and pyelftools to look up what it needs, plus judgment about when each tool is appropriate.

Scope Boundaries

Prescriptiveness should match task risk:

• Strict for fragile tasks - Security audits, crypto implementations, compliance checks need rigid step-by-step enforcement
• Flexible for variable tasks - Code exploration, documentation, refactoring can offer options and judgment calls

Security Skills

For audit/security skills, also include:

## Rationalizations to Reject
[Common shortcuts or rationalizations that lead to missed findings]

Content Organization

• Keep SKILL.md under 500 lines - split into references/, workflows/
• Use progressive disclosure - quick start first, details in linked files
• One level deep - SKILL.md links to files, files don't chain to more files

Note: Directory depth is fine (references/guides/topic.md). Reference chains are not (SKILL.md → file1.md → file2.md where file1 references file2). The problem is chained references, not nested folders.

Progressive Disclosure Pattern

## Quick Start
[Core instructions here]

## Advanced Usage
See [ADVANCED.md](references/ADVANCED.md) for detailed patterns.

## API Reference
See [API.md](references/API.md) for complete method documentation.

Before committing

make check

That runs the validator self-test, ruff, shellcheck, shfmt, bats, the plugin Python suites, and the plugin validator.

It is most of CI, not all of it. Three things run only in CI, so a green make check is strong evidence and not a guarantee:

• the two loadability checks, which need the Claude Code and Codex CLIs installed
• the rest of pre-commit — actionlint, zizmor, check-yaml/json/toml, detect-private-key, end-of-file-fixer, trailing-whitespace. Run prek run -a (or pre-commit run -a) to cover those locally.
• the version-increment check, which needs a base ref to diff against and so has no meaning outside a PR.
• make shell-suites, which is a target but not part of check: it still fails on a machine with the modern-python plugin installed, though no longer for the reason #207 describes. The python3 - interception that broke zeroize-audit is gone as of modern-python 1.6.0. What remains is plugins/variant-analysis/tests/ invoking python3 <script>.py, which the shim intercepts by design — a bare script run is exactly what uv run python replaces. That one is variant-analysis's to fix. With no shim on PATH the whole target passes.

Both scan every plugin; the validator is not scoped down in CI. Only the version-increment check is limited to the plugins a branch touched, and it is the one check CI runs that local cannot. Do not add a scoping flag to the local run — the zero-reference guard only arms on a full scan.

make fix applies the formatting CI would otherwise reject. make help lists the rest.

What the validator enforces, so you do not have to

Each of these fails the build. There is no value in checking any of it by hand:

• plugin.json exists, parses, and has name, description, and a semver version
• plugin.json's name equals the plugin's directory name
• Plugin directory name is kebab-case and ≤64 characters
• Plugin has a README.md (exact case — Readme.md passes on macOS and fails on CI)
• Registered in .claude-plugin/marketplace.json, the root README.md, and CODEOWNERS
• The marketplace entry's source is exactly ./plugins/<name> and its description matches plugin.json
• version matches between plugin.json and marketplace.json, and increases when you change a plugin — clients only pull an update when the number goes up, so a fix shipped without a bump reaches nobody. Apply the no-version-bump label for typo-only changes and CI skips the check.
• SKILL.md has frontmatter, and no top-level value is an unquoted YAML scalar containing : or #. Either one makes the whole block unparseable, and the loader then drops every field and loads the skill with empty metadata — so a skill whose description: line reads perfectly well ships with no description and never triggers. Quote any description containing a colon.
• Agent files use tools:; skills and commands use allowed-tools:
• subagent_type values are namespaced <plugin>:<agent> — a bare name is unregistered and the dispatch fails at runtime, whether it names this plugin's own agent, another plugin's, or nothing at all
• No hardcoded /Users/… or /home/… paths, in any .md, .py, .json, .sh, .bats, .yml or .toml file under plugins/. *-shim.bats is exempt because those fixtures need literal paths, and /path/to and /home/vscode are treated as placeholders rather than somebody's home directory.
• No .codex/, .opencode/, .agents/, or plugins/*/.codex-plugin/ sidecars
• A committed uv.lock for every uv directory listed in .github/dependabot.yml
• Both loadability checks pass under the real Claude Code and Codex CLIs

Two more are reported as warnings, so they will not stop a merge and do still need your eye: SKILL.md over 500 lines, and references that do not resolve. A dangling references/setup.md link 404s for every user of the skill, and CI will not stop you shipping it.

What no tool can check — this is the part that needs you

• The description actually triggers. Third person, names the situation, uses the words a user would actually type. This is the single highest-leverage line in a skill: a skill that never triggers may as well not exist.
• Examples are concrete — real input, real output, not a shape.
• It explains why, including the trade-off and when not to do the thing.
• The version bump is the right size. The validator confirms the number went up; only you know whether the change was substantive. MAJOR.MINOR.PATCH, MINOR for features, PATCH for fixes.
• CODEOWNERS lists the right people: /plugins/<name>/ @you @dguido. Find your username with gh api user --jq .login.
• The entry is in the right section of the root README.md. The validator only checks that the plugin appears somewhere in that file, so a row appended to whichever table you scrolled to first passes CI and stays filed under the wrong category indefinitely.

Scripts a plugin ships

A checker that inspects zero items must fail, not pass. This is the single most expensive class of bug in a repo like this one, because it is invisible on every read and in every review. Real examples, all of which were green for months:

• a validator using grep -oP (rejected by BSD grep) with stderr sent to /dev/null, so it printed "all valid" on every run without ever matching anything
• an eval grader that judged the response text rather than the artifact, so a run that skipped the actual work still scored a pass
• a citation gate that validated only the citations that were present, so a document with zero citations passed

If your script counts, filters, or matches, make it exit non-zero when the count is zero, and give it a fixture proving it still detects its target. The validator's --self-test is the worked example: it builds a known-bad plugin in a tempdir and asserts each checker rejects it, and it fails if it runs fewer assertions than it should — because the self-test is itself a checker.

Otherwise: set -euo pipefail, POSIX ERE rather than PCRE (grep -oP is not portable to macOS), and never send a tool's stderr to /dev/null unless you have handled the failure it would have reported.

Working effectively in this repo

• Effort. Start at xhigh for coding and agentic work and high elsewhere, then sweep downward on your own evals — low and medium are unusually strong on current models and often match what an older model needed xhigh to do. Defaults carried over from a previous model are rarely the right setting.
• Subagents. Delegate work that is large and genuinely independent — a wide multi-file investigation, several unrelated tracks. Do not spawn subagents to verify or double-check your own work, and do not split one modest job across several: each one re-establishes context, re-explores, and reports back, and then you re-read the report. One well-briefed agent beats three vague ones.
• Do not add verification scaffolding to prompts. "Double-check your answer", "add a final verification step", and similar make output worse on current models rather than better — they cause over-verification, and removing them costs no capability. This inverts older advice, so it is worth stating explicitly. Put the check in make check or the validator, where it runs deterministically and cannot be talked out of firing. This applies to skills you write or edit; existing skills carrying the pattern are not a cleanup backlog, so strip it when you are already in the file rather than as its own sweep.
• Do not tell a reviewer to pre-filter. "Only report high-severity issues" is followed literally: the model investigates just as thoroughly, finds the bugs, and then declines to report what it judges below the bar. Precision rises, recall appears to collapse, and the regression looks like a capability problem when it is a prompt problem. Ask for everything with a severity attached and filter in a separate pass — c-review and rust-review do this correctly if you want a model to copy.