mirror of
https://github.com/trailofbits/skills.git
synced 2026-09-14 14:28:48 +08:00
kz-tob
4b1b74b181
* Give differential-review a trigger, and name every component in its README differential-review's description listed what it does and never named a situation, so it competed on capability wording alone. It now closes with the triggers its own README already documents — reviewing a PR, commit, or diff; checking whether a change re-introduces a fixed bug; asking what else a change could break; finding modified code with no test. The same plugin's README never mentioned adversarial-modeler, which is what Phase 5 dispatches for HIGH RISK changes. Checking whether that was isolated turned up more of it, and the sweep found three kinds of gap: zeroize-audit's agent table was missing three of its eleven agents — 0-preflight, which gates the entire run, plus 5b-poc-validator and 5c-poc-verifier. All three appear in the phase diagram directly above the table, which is why they read as present. constant-time-analysis documents the ct-analyzer CLI end to end and never says the plugin also ships a skill and a command. entry-point-analyzer lists phrases that trigger its skill but never names the skill or its command. Three more READMEs describe their skill without naming it. That matters most where the skill name is not the plugin name and a user cannot guess it: chrome-mcp-troubleshooting and interpreting-culture-index. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix review findings and make the README sweep a gate The two PoC rows I added to zeroize-audit said Phase 4. The diagram three lines above them, SKILL.md, and workflows/phase-5-poc-validation.md all say Phase 5, steps 5a and 5b. "Wave 5a" is a label that exists nowhere. A debugger consulting the table — the artifact this branch designates as what runs when — would have opened phase-4-poc-generation.md and found no validation in it. Also corrected the sentence introducing that table, which still said 10 agents across 8 phases against 11 across 9, and the Phase 0 diagram line, which still credited the orchestrator for a gate the new row credits to 0-preflight. differential-review's README claimed the agent is "dispatched", and named it bare in a column whose other rows are namespaced. Nothing dispatches it: the only instruction is prose in SKILL.md, and a bare subagent_type fails at runtime. Namespaced both, and corrected the five stale line counts in the same file — reporting.md is 369 lines, not the ~120 the token-efficiency section budgets for. Drop the dead `name: trailofbits:<cmd>` key from five command files. The three newest command files carry no name: at all, #275 namespaced 22 bare invocations, and this branch documents the `/<plugin>:<cmd>` form — so the key contradicts the docs it sits next to. Then make the sweep repeatable. Doing this by hand three times found eight gaps and missed two more, both of the same shape: a workflow ships under meta.name, not its filename, so a README citing the filename never writes the name a reader types. The validator now checks that a README names every skill, agent, command, and workflow its plugin ships, reading meta.name for workflows. It refuses a run that inspected zero components, and six self-test assertions hold it to known-bad fixtures. It found git-cleanup on its first run: ships as /git-cleanup:git-cleanup-analysis, README cites workflows/analyze-branches.js four times and that name never. static-analysis had the same gap for codeql-build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix both P2s: the gate was a substring test, and the dispatch was still bare The README gate ran `name not in text`. That reads as thorough and could not fail for a large share of what it counted: `draw` was satisfied by "(draw cards instead)", `semgrep-rule` by the plugin's own name in the install line, `burp-search` by a `scripts/burp-search.sh` path that is a different thing, and `audit` by the prose "shared-state struct audit". Match by kind instead. Commands and workflows are reachable only as `/<plugin>:<name>`, so require that literal — it is the only string a user can type. Agents are dispatched by identifier and never typed as prose, so require an identifier-shaped mention. Skills are genuinely referred to by bare name, so require only a delimited occurrence, which is what stops "draws" counting as `draw`. That surfaced seven real gaps, the four above plus insecure-defaults' audit-pipeline workflow, mutation-testing's skill, and trailmark's code-slice-worker. All seven fixed. adversarial-modeler was still bare at SKILL.md:96. Line 77 was the decision-tree mention; line 96 is the "Delegate to this agent" instruction a model actually acts on, so the runtime failure the last commit claimed to fix survived it. Namespaced, and it now says why. Also from the review: a per-kind floor, since a single total stays healthy while skill_files() — 63% of coverage — silently stops matching; workflow_names anchored to the meta block, because a bare search takes any earlier `name:` in a comment, and .mjs was invisible; and AGENTS.md documents the new hard failure. Self-test 88 -> 96, each new rule with a negative control. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Differential Review
Security-focused differential review of code changes with git history analysis and blast radius estimation.
Author: Omar Inuwa
When to Use
Use this skill when you need to:
- Review PRs, commits, or diffs for security vulnerabilities
- Detect security regressions (re-introduced vulnerabilities)
- Analyze the blast radius of code changes
- Check test coverage gaps for modified code
What It Does
This skill performs comprehensive security review of code changes:
- Risk-First Analysis - Prioritizes auth, crypto, value transfer, external calls
- Git History Analysis - Uses blame to understand why code existed and detect regressions
- Blast Radius Calculation - Quantifies impact by counting callers
- Test Coverage Gaps - Identifies untested changes
- Adaptive Depth - Scales analysis based on codebase size (small/medium/large)
Installation
/plugin install trailofbits/skills/plugins/differential-review
Components
| Component | Name | Purpose |
|---|---|---|
| Skill | differential-review |
The review itself — Pre-Analysis plus Phases 0–6, described below |
| Command | /differential-review:diff-review |
Entry point taking <pr-url|commit-sha|diff-path> [--baseline <ref>] |
| Agent | differential-review:adversarial-modeler |
Phase 5 only. The skill delegates to it for HIGH RISK changes to build concrete exploit paths and rate exploitability |
Documentation Structure
This skill uses a modular documentation architecture for token efficiency and progressive disclosure:
Core Entry Point
- SKILL.md - Main entry point (224 lines)
- Quick reference tables for triage
- Decision tree routing to detailed docs
- Quality checklist and red flags
- Integration with other skills
Supporting Documentation
-
methodology.md - Detailed phase-by-phase workflow (234 lines)
- Pre-Analysis: Baseline context building
- Phase 0: Intake & Triage
- Phase 1: Changed Code Analysis
- Phase 2: Test Coverage Analysis
- Phase 3: Blast Radius Analysis
- Phase 4: Deep Context Analysis
-
adversarial.md - Attacker modeling and exploit scenarios (203 lines)
- Phase 5: Adversarial Vulnerability Analysis
- Attacker model definition (WHO/ACCESS/INTERFACE)
- Exploitability rating framework
- Complete exploit scenario templates
-
reporting.md - Report structure and formatting (369 lines)
- Phase 6: Report Generation
- 9-section report template
- Formatting guidelines and conventions
- File naming and notification templates
-
patterns.md - Common vulnerability patterns (300 lines)
- Security regressions detection
- Reentrancy, access control, overflow patterns
- Quick detection bash commands
Benefits of This Structure
- Token Efficient - Load only the documentation you need
- Progressive Disclosure - Quick reference for triage, detailed docs for deep analysis
- Maintainable - Each concern separated into its own file
- Navigable - Decision tree routes you to the right document
Workflow
The complete workflow spans Pre-Analysis + Phases 0-6:
- Pre-Analysis - Build baseline context with
audit-context-buildingskill (if available) - Phase 0: Intake - Extract changes, assess size, risk-score files
- Phase 1: Changed Code - Analyze diffs, git blame, check for regressions
- Phase 2: Test Coverage - Identify coverage gaps
- Phase 3: Blast Radius - Calculate impact of changes
- Phase 4: Deep Context - Five Whys root cause analysis
- Phase 5: Adversarial Analysis - Hunt vulnerabilities with attacker model
- Phase 6: Report - Generate comprehensive markdown report
Navigation: Use the decision tree in SKILL.md to jump directly to the phase you need.
Output
Generates a markdown report with:
- Executive summary with severity distribution
- Critical findings with attack scenarios and PoCs
- Test coverage analysis
- Blast radius analysis
- Historical context and regression risks
- Actionable recommendations
Example Usage
Review the security implications of this PR:
git diff main..feature/auth-changes
Related Skills
context-building- Used for baseline context analysisissue-writer- Transform findings into formal audit reports