mirror of
https://github.com/trailofbits/skills.git
synced 2026-09-14 14:28:48 +08:00
Give differential-review a trigger, and name every component in its README (#278)
* Give differential-review a trigger, and name every component in its README differential-review's description listed what it does and never named a situation, so it competed on capability wording alone. It now closes with the triggers its own README already documents — reviewing a PR, commit, or diff; checking whether a change re-introduces a fixed bug; asking what else a change could break; finding modified code with no test. The same plugin's README never mentioned adversarial-modeler, which is what Phase 5 dispatches for HIGH RISK changes. Checking whether that was isolated turned up more of it, and the sweep found three kinds of gap: zeroize-audit's agent table was missing three of its eleven agents — 0-preflight, which gates the entire run, plus 5b-poc-validator and 5c-poc-verifier. All three appear in the phase diagram directly above the table, which is why they read as present. constant-time-analysis documents the ct-analyzer CLI end to end and never says the plugin also ships a skill and a command. entry-point-analyzer lists phrases that trigger its skill but never names the skill or its command. Three more READMEs describe their skill without naming it. That matters most where the skill name is not the plugin name and a user cannot guess it: chrome-mcp-troubleshooting and interpreting-culture-index. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix review findings and make the README sweep a gate The two PoC rows I added to zeroize-audit said Phase 4. The diagram three lines above them, SKILL.md, and workflows/phase-5-poc-validation.md all say Phase 5, steps 5a and 5b. "Wave 5a" is a label that exists nowhere. A debugger consulting the table — the artifact this branch designates as what runs when — would have opened phase-4-poc-generation.md and found no validation in it. Also corrected the sentence introducing that table, which still said 10 agents across 8 phases against 11 across 9, and the Phase 0 diagram line, which still credited the orchestrator for a gate the new row credits to 0-preflight. differential-review's README claimed the agent is "dispatched", and named it bare in a column whose other rows are namespaced. Nothing dispatches it: the only instruction is prose in SKILL.md, and a bare subagent_type fails at runtime. Namespaced both, and corrected the five stale line counts in the same file — reporting.md is 369 lines, not the ~120 the token-efficiency section budgets for. Drop the dead `name: trailofbits:<cmd>` key from five command files. The three newest command files carry no name: at all, #275 namespaced 22 bare invocations, and this branch documents the `/<plugin>:<cmd>` form — so the key contradicts the docs it sits next to. Then make the sweep repeatable. Doing this by hand three times found eight gaps and missed two more, both of the same shape: a workflow ships under meta.name, not its filename, so a README citing the filename never writes the name a reader types. The validator now checks that a README names every skill, agent, command, and workflow its plugin ships, reading meta.name for workflows. It refuses a run that inspected zero components, and six self-test assertions hold it to known-bad fixtures. It found git-cleanup on its first run: ships as /git-cleanup:git-cleanup-analysis, README cites workflows/analyze-branches.js four times and that name never. static-analysis had the same gap for codeql-build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix both P2s: the gate was a substring test, and the dispatch was still bare The README gate ran `name not in text`. That reads as thorough and could not fail for a large share of what it counted: `draw` was satisfied by "(draw cards instead)", `semgrep-rule` by the plugin's own name in the install line, `burp-search` by a `scripts/burp-search.sh` path that is a different thing, and `audit` by the prose "shared-state struct audit". Match by kind instead. Commands and workflows are reachable only as `/<plugin>:<name>`, so require that literal — it is the only string a user can type. Agents are dispatched by identifier and never typed as prose, so require an identifier-shaped mention. Skills are genuinely referred to by bare name, so require only a delimited occurrence, which is what stops "draws" counting as `draw`. That surfaced seven real gaps, the four above plus insecure-defaults' audit-pipeline workflow, mutation-testing's skill, and trailmark's code-slice-worker. All seven fixed. adversarial-modeler was still bare at SKILL.md:96. Line 77 was the decision-tree mention; line 96 is the "Delegate to this agent" instruction a model actually acts on, so the runtime failure the last commit claimed to fix survived it. Namespaced, and it now says why. Also from the review: a per-kind floor, since a single total stays healthy while skill_files() — 63% of coverage — silently stops matching; workflow_names anchored to the meta block, because a bare search takes any earlier `name:` in a comment, and .mjs was invisible; and AGENTS.md documents the new hard failure. Self-test 88 -> 96, each new rule with a negative control. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kz-tob
@@ -29,7 +29,7 @@
|
||||
},
|
||||
{
|
||||
"name": "burpsuite-project-parser",
|
||||
"version": "1.0.1",
|
||||
"version": "1.0.2",
|
||||
"description": "Search and extract data from Burp Suite project files (.burp) for security analysis",
|
||||
"author": {
|
||||
"name": "Will Vandevanter"
|
||||
@@ -38,7 +38,7 @@
|
||||
},
|
||||
{
|
||||
"name": "claude-in-chrome-troubleshooting",
|
||||
"version": "1.1.1",
|
||||
"version": "1.1.2",
|
||||
"description": "Diagnose and fix Claude in Chrome MCP extension connectivity issues",
|
||||
"author": {
|
||||
"name": "Dan Guido"
|
||||
@@ -47,7 +47,7 @@
|
||||
},
|
||||
{
|
||||
"name": "constant-time-analysis",
|
||||
"version": "0.2.2",
|
||||
"version": "0.2.3",
|
||||
"description": "Detect compiler-induced timing side-channels in cryptographic code",
|
||||
"author": {
|
||||
"name": "Scott Arciszewski",
|
||||
@@ -57,7 +57,7 @@
|
||||
},
|
||||
{
|
||||
"name": "culture-index",
|
||||
"version": "1.1.2",
|
||||
"version": "1.1.3",
|
||||
"description": "Interprets Culture Index survey results for individuals and teams",
|
||||
"author": {
|
||||
"name": "Dan Guido"
|
||||
@@ -66,7 +66,7 @@
|
||||
},
|
||||
{
|
||||
"name": "devcontainer-setup",
|
||||
"version": "0.2.1",
|
||||
"version": "0.2.2",
|
||||
"description": "Create pre-configured devcontainers with Claude Code and language-specific tooling",
|
||||
"author": {
|
||||
"name": "Alexis Challande",
|
||||
@@ -77,7 +77,7 @@
|
||||
{
|
||||
"name": "differential-review",
|
||||
"description": "Security-focused differential review of code changes with git history analysis and blast radius estimation",
|
||||
"version": "1.1.1",
|
||||
"version": "1.1.2",
|
||||
"author": {
|
||||
"name": "Omar Inuwa"
|
||||
},
|
||||
@@ -115,7 +115,7 @@
|
||||
},
|
||||
{
|
||||
"name": "entry-point-analyzer",
|
||||
"version": "1.0.1",
|
||||
"version": "1.0.2",
|
||||
"description": "Analyzes smart contract codebases to identify state-changing entry points for security auditing. Detects externally callable functions that modify state, categorizes them by access level, and generates structured audit reports.",
|
||||
"author": {
|
||||
"name": "Nicolas Donboly",
|
||||
@@ -126,7 +126,7 @@
|
||||
},
|
||||
{
|
||||
"name": "mutation-testing",
|
||||
"version": "1.0.1",
|
||||
"version": "1.0.2",
|
||||
"description": "Configures mewt or muton mutation testing campaigns — scopes targets, tunes timeouts, and optimizes long-running runs. Use when the user mentions mewt, muton, mutation testing, or wants to configure or optimize a mutation testing campaign.",
|
||||
"author": {
|
||||
"name": "Trail of Bits",
|
||||
@@ -147,7 +147,7 @@
|
||||
},
|
||||
{
|
||||
"name": "semgrep-rule-creator",
|
||||
"version": "1.2.3",
|
||||
"version": "1.2.4",
|
||||
"description": "Create custom Semgrep rules for detecting bug patterns and security vulnerabilities",
|
||||
"author": {
|
||||
"name": "Maciej Domanski"
|
||||
@@ -177,7 +177,7 @@
|
||||
},
|
||||
{
|
||||
"name": "static-analysis",
|
||||
"version": "1.4.0",
|
||||
"version": "1.4.1",
|
||||
"description": "Static analysis toolkit with CodeQL, Semgrep, and SARIF parsing for security vulnerability detection",
|
||||
"author": {
|
||||
"name": "Axel Mierczuk & Paweł Płatek"
|
||||
@@ -204,7 +204,7 @@
|
||||
},
|
||||
{
|
||||
"name": "trailmark",
|
||||
"version": "0.11.1",
|
||||
"version": "0.11.2",
|
||||
"description": "Builds multi-language source and binary code graphs for security analysis: call graphs, attack surface mapping, blast radius, taint propagation, complexity hotspots, entry point enumeration, proxy/unresolved-call tracking, type/reference analysis, and structural diffs. Creates bounded, graph-informed source packets for delegating focused work to constrained subagents. Generates Mermaid diagrams, runs graph-informed mutation testing triage (genotoxic), generates mutation-driven test vectors (vector-forge), extracts crypto protocol message flows, converts Mermaid diagrams to ProVerif models, projects SARIF/weAudit/binary findings onto code graphs, triages single findings with graph evidence, gates branch diffs for structural review regressions, and expands seed findings into variant-neighborhood candidates. Use when analyzing call paths, slicing source context for smaller models, mapping attack surface, visualizing code architecture, triaging survived mutants, generating cryptographic test vectors, diagramming crypto protocols, formally verifying protocols, augmenting audits with static analysis findings, deciding whether one candidate issue is reachable, reviewing graph-level PR risk, or seeding variant analysis.",
|
||||
"author": {
|
||||
"name": "Scott Arciszewski",
|
||||
@@ -223,7 +223,7 @@
|
||||
},
|
||||
{
|
||||
"name": "c-review",
|
||||
"version": "2.0.0",
|
||||
"version": "2.0.1",
|
||||
"description": "Comprehensive C/C++ security code review, with coverage verified against a parse of the source",
|
||||
"author": {
|
||||
"name": "Paweł Płatek"
|
||||
@@ -252,7 +252,7 @@
|
||||
},
|
||||
{
|
||||
"name": "insecure-defaults",
|
||||
"version": "2.0.0",
|
||||
"version": "2.0.1",
|
||||
"description": "Detects insecure default configurations including hardcoded credentials, fallback secrets, weak authentication defaults, and dangerous values in production",
|
||||
"author": {
|
||||
"name": "Trail of Bits",
|
||||
@@ -283,7 +283,7 @@
|
||||
},
|
||||
{
|
||||
"name": "git-cleanup",
|
||||
"version": "2.3.0",
|
||||
"version": "2.3.1",
|
||||
"description": "Safely analyzes and cleans up local git branches and worktrees by categorizing them as merged, squash-merged, superseded, or active work.",
|
||||
"author": {
|
||||
"name": "Henrik Brodin",
|
||||
@@ -312,7 +312,7 @@
|
||||
},
|
||||
{
|
||||
"name": "zeroize-audit",
|
||||
"version": "0.3.0",
|
||||
"version": "0.3.1",
|
||||
"description": "Detects missing or compiler-optimized zeroization of sensitive data with assembly and control-flow analysis",
|
||||
"author": {
|
||||
"name": "Trail of Bits",
|
||||
@@ -323,7 +323,7 @@
|
||||
},
|
||||
{
|
||||
"name": "let-fate-decide",
|
||||
"version": "1.2.2",
|
||||
"version": "1.2.3",
|
||||
"description": "Draws the 12 Houses of the Zodiac Tarot spread using cryptographic randomness to add 100+ bits of entropy to vague or underspecified planning. Interprets the spread to guide next steps. Use when feeling lucky, invoking heart-of-the-cards energy, or when prompts are ambiguous.",
|
||||
"author": {
|
||||
"name": "Scott Arciszewski",
|
||||
|
||||
@@ -164,7 +164,7 @@ SCAN_SKIP_DIRS = frozenset({".venv", "venv", "node_modules", "__pycache__", ".gi
|
||||
|
||||
# Floor for --self-test, set to the exact number of assertions the fixtures run. There is
|
||||
# no slack on purpose: dropping one has to be a deliberate edit here, not a silent loss.
|
||||
SELF_TEST_MINIMUM = 82
|
||||
SELF_TEST_MINIMUM = 96
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -187,6 +187,8 @@ class ScanResult:
|
||||
refs_checked: int = 0
|
||||
paths_scanned: int = 0
|
||||
python_docs_scanned: int = 0
|
||||
components_checked: int = 0
|
||||
components_by_kind: dict[str, int] = field(default_factory=dict)
|
||||
|
||||
def add(self, plugin: str, message: str, severity: str = ERROR) -> None:
|
||||
self.findings.append(Finding(plugin, message, severity))
|
||||
@@ -512,6 +514,102 @@ def validate_entry_points(plugin_path: Path) -> list[str]:
|
||||
return ["exposes no entry point: no skills/, commands/, agents/, hooks, or .mcp.json"]
|
||||
|
||||
|
||||
def workflow_names(plugin_path: Path) -> list[tuple[str, str]]:
|
||||
"""Dynamic workflows a plugin ships, as (display path, invocable name) pairs.
|
||||
|
||||
A workflow ships as `/<plugin>:<meta.name>`, and `meta.name` is frequently not the
|
||||
filename — so a scan that globbed filenames would look thorough and still miss the
|
||||
name a reader has to type. Falls back to the stem only when `meta.name` is absent.
|
||||
"""
|
||||
workflows_dir = plugin_path / "workflows"
|
||||
if not workflows_dir.is_dir():
|
||||
return []
|
||||
found = []
|
||||
for path in sorted(workflows_dir.rglob("*.[mc]js")) + sorted(workflows_dir.rglob("*.js")):
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
# Anchor to the meta block. A bare search takes the first `name:` in the file,
|
||||
# and these scripts carry `name:` inside comments and inside agent option
|
||||
# objects — code-improver's improve.js already has one in a comment. A file
|
||||
# with no `export const meta` is a helper, not a shippable workflow.
|
||||
start = text.find("export const meta")
|
||||
if start == -1:
|
||||
continue
|
||||
match = re.search(r"""\bname:\s*['"]([^'"]+)['"]""", text[start:])
|
||||
if not match:
|
||||
continue # A computed or templated name is not something a README can quote.
|
||||
found.append((path.name, match.group(1)))
|
||||
return sorted(set(found))
|
||||
|
||||
|
||||
def _readme_names(text: str, kind: str, name: str, plugin: str) -> bool:
|
||||
"""Whether a README names one component, as opposed to merely containing its letters.
|
||||
|
||||
A plain `name in text` looks thorough and cannot fail for a large share of what it
|
||||
counts: `draw` is satisfied by "draws Tarot cards", `semgrep-rule` by the plugin's
|
||||
own name in the install line, and `burp-search` by a `scripts/burp-search.sh` path
|
||||
that is a different thing entirely.
|
||||
|
||||
Commands and workflows are reachable only as `/<plugin>:<name>`, so that literal is
|
||||
the only mention that helps a reader — nothing else tells them what to type.
|
||||
|
||||
Agents are dispatched by identifier and never typed as prose, so a bare word in a
|
||||
sentence does not name one: `let-fate-decide`'s `draw` agent was satisfied by
|
||||
"(draw cards instead)". They need an identifier-shaped mention — backticked, or as
|
||||
an `agents/<name>` path, or namespaced.
|
||||
|
||||
Skills are the one kind genuinely referred to by bare name in prose and tables, so
|
||||
they need only a delimited occurrence — one not glued to a longer identifier, which
|
||||
is what stops "draws" counting as `draw` and `semgrep-rule-creator` as `semgrep-rule`.
|
||||
"""
|
||||
if kind in ("command", "workflow"):
|
||||
return f"/{plugin}:{name}" in text
|
||||
if kind == "agent":
|
||||
return any(form in text for form in (f"`{name}`", f"agents/{name}", f"{plugin}:{name}"))
|
||||
delimited = re.compile(rf"(?<![A-Za-z0-9_-]){re.escape(name)}(?![A-Za-z0-9_-])")
|
||||
return delimited.search(text) is not None
|
||||
|
||||
|
||||
def validate_readme_names_components(plugin_path: Path) -> tuple[list[str], int]:
|
||||
"""A plugin's README must name every component the plugin ships.
|
||||
|
||||
A README that describes a skill without naming it leaves the reader unable to invoke
|
||||
it, and the gap is worst exactly where it is least guessable — when the skill name is
|
||||
not the plugin name. The same applies to agents, commands, and workflows: an agent
|
||||
missing from a pipeline table reads as a pipeline that does not have it.
|
||||
|
||||
Returns the findings, the number of components inspected, and that count broken down
|
||||
by kind. The counts are returned so the caller can refuse a run that inspected
|
||||
nothing — a sweep over zero components reports "all clean" exactly like a sweep over
|
||||
all of them — and per-kind so the loss of one discovery helper cannot hide inside a
|
||||
healthy total.
|
||||
"""
|
||||
readme = plugin_path / "README.md"
|
||||
if not readme.is_file():
|
||||
return [], 0, {} # A missing README is already an error elsewhere.
|
||||
text = readme.read_text(encoding="utf-8", errors="replace")
|
||||
|
||||
components: list[tuple[str, str]] = []
|
||||
components += [("skill", p.parent.name) for p in skill_files(plugin_path)]
|
||||
components += [("agent", p.stem) for p in agent_files(plugin_path)]
|
||||
components += [("command", p.stem) for p in command_files(plugin_path)]
|
||||
components += [("workflow", name) for _, name in workflow_names(plugin_path)]
|
||||
|
||||
findings = []
|
||||
for kind, name in components:
|
||||
if _readme_names(text, kind, name, plugin_path.name):
|
||||
continue
|
||||
wanted = f"/{plugin_path.name}:{name}" if kind in ("command", "workflow") else f"'{name}'"
|
||||
findings.append(
|
||||
f"README.md does not name the {kind} {wanted} — "
|
||||
f"a reader cannot invoke what is not named"
|
||||
)
|
||||
|
||||
by_kind: dict[str, int] = {}
|
||||
for kind, _ in components:
|
||||
by_kind[kind] = by_kind.get(kind, 0) + 1
|
||||
return findings, len(components), by_kind
|
||||
|
||||
|
||||
def validate_subagent_dispatch(
|
||||
plugin_path: Path,
|
||||
plugin_name: str,
|
||||
@@ -1000,6 +1098,13 @@ def validate_plugins(
|
||||
for msg in validate_subagent_dispatch(plugin_path, plugin_name, agent_owners):
|
||||
result.add(plugin_name, msg)
|
||||
|
||||
readme_findings, components, by_kind = validate_readme_names_components(plugin_path)
|
||||
result.components_checked += components
|
||||
for kind, count in by_kind.items():
|
||||
result.components_by_kind[kind] = result.components_by_kind.get(kind, 0) + count
|
||||
for msg in readme_findings:
|
||||
result.add(plugin_name, msg)
|
||||
|
||||
if base_ref and plugin_name in version_check_scope:
|
||||
for msg in validate_version_increment(repo_root, plugin_name, plugin_data, base_ref):
|
||||
result.add(plugin_name, msg)
|
||||
@@ -1091,6 +1196,20 @@ def main(argv: list[str] | None = None) -> int:
|
||||
print("\n✗ legacy-python scan read no files at all — discovery is broken")
|
||||
return 1
|
||||
|
||||
# Four independent discovery sources feed this. A single total would stay comfortably
|
||||
# non-zero if skill_files() — 81 of 129 components — stopped matching, so each kind
|
||||
# carries its own floor: this repo ships all four, so a zero anywhere is a broken
|
||||
# glob, not an empty category.
|
||||
seen = result.components_by_kind
|
||||
empty_kinds = [kind for kind, count in sorted(seen.items()) if count == 0]
|
||||
missing_kinds = sorted({"skill", "agent", "command", "workflow"} - set(seen))
|
||||
if empty_kinds or missing_kinds:
|
||||
broken = ", ".join(empty_kinds + missing_kinds)
|
||||
print(
|
||||
f"\n✗ README component scan found no {broken} components at all — discovery is broken"
|
||||
)
|
||||
return 1
|
||||
|
||||
errors = [f for f in result.findings if f.severity == ERROR]
|
||||
if errors:
|
||||
return 1
|
||||
@@ -1098,7 +1217,8 @@ def main(argv: list[str] | None = None) -> int:
|
||||
print(
|
||||
f"\n✓ no errors ({result.refs_checked} references resolved, "
|
||||
f"{result.paths_scanned} files scanned for hardcoded paths, "
|
||||
f"{result.python_docs_scanned} for legacy python invocations)"
|
||||
f"{result.python_docs_scanned} for legacy python invocations, "
|
||||
f"{result.components_checked} components named in their README)"
|
||||
)
|
||||
return 0
|
||||
|
||||
@@ -1124,7 +1244,16 @@ def _build_demo(root: Path, name: str = "demo") -> Path:
|
||||
}
|
||||
),
|
||||
)
|
||||
_write(plugin / "README.md", f"# {name}\n")
|
||||
# Names the component vocabulary the other fixtures use, so each of them stays
|
||||
# isolated to the checker it targets instead of also tripping the
|
||||
# README-names-its-components check. _self_test_readme_components overwrites
|
||||
# this when a missing name is the thing under test.
|
||||
_write(
|
||||
plugin / "README.md",
|
||||
f"# {name}\n\nCommands and workflows: `/{name}:go`, `/{name}:run-it`, "
|
||||
f"`/{name}:audit`, `/{name}:demo-analysis`.\n\n"
|
||||
"Agents: `worker`, `helper`, `w`.\n",
|
||||
)
|
||||
_write(
|
||||
plugin / "skills" / name / "SKILL.md",
|
||||
"---\nname: demo\ndescription: Demo.\nallowed-tools: Read Grep\n---\n\n"
|
||||
@@ -1559,6 +1688,135 @@ def _self_test_errors(ran: list[str]) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _self_test_readme_components(ran: list[str]) -> None:
|
||||
"""The README-names-its-components checker fires, and does not over-fire."""
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
_build_demo(root)
|
||||
_check(
|
||||
ran,
|
||||
"README naming its skill is accepted",
|
||||
not any("does not name the skill" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(plugin / "README.md", "# A plugin\n\nIt reviews things.\n")
|
||||
_check(
|
||||
ran,
|
||||
"skill absent from README",
|
||||
any("does not name the skill 'demo'" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(
|
||||
plugin / "agents" / "helper.md",
|
||||
"---\nname: helper\ndescription: x\ntools:\n - Read\n---\n",
|
||||
)
|
||||
_write(plugin / "README.md", "# demo\n\nIt does things.\n")
|
||||
_check(
|
||||
ran,
|
||||
"agent absent from README",
|
||||
any("does not name the agent 'helper'" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(plugin / "commands" / "run-it.md", "---\ndescription: x\n---\n\n# Run\n")
|
||||
_write(plugin / "README.md", "# demo\n\nIt does things.\n")
|
||||
_check(
|
||||
ran,
|
||||
"command absent from README",
|
||||
any("does not name the command /demo:run-it" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(plugin / "commands" / "go.md", "---\ndescription: x\nallowed-tools: Read\n---\n")
|
||||
_write(plugin / "README.md", "# demo\n\nRun `scripts/go.sh` to go.\n")
|
||||
_check(
|
||||
ran,
|
||||
"command satisfied only by a same-named script path",
|
||||
any("does not name the command /demo:go" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(plugin / "commands" / "go.md", "---\ndescription: x\nallowed-tools: Read\n---\n")
|
||||
_write(plugin / "README.md", "# demo\n\nInvoke `/demo:go` to go.\n")
|
||||
_check(
|
||||
ran,
|
||||
"command named by its slash form is accepted",
|
||||
not any("does not name the command" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
# A bare word in a sentence is not a dispatchable identifier. This is the shape that
|
||||
# let let-fate-decide's `draw` agent pass on "(draw cards instead)".
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(plugin / "agents" / "w.md", "---\nname: w\ndescription: x\ntools:\n - Read\n---\n")
|
||||
_write(plugin / "README.md", "# demo\n\nIt goes w places, w times over.\n")
|
||||
_check(
|
||||
ran,
|
||||
"agent satisfied only by prose",
|
||||
any("does not name the agent 'w'" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(plugin / "agents" / "w.md", "---\nname: w\ndescription: x\ntools:\n - Read\n---\n")
|
||||
_write(plugin / "README.md", "# demo\n\nDispatch `w` for that.\n")
|
||||
_check(
|
||||
ran,
|
||||
"agent named in identifier form is accepted",
|
||||
not any("does not name the agent" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
# A skill name that is only ever a prefix of a longer identifier is not named.
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root, name="demo")
|
||||
(plugin / "skills" / "demo").rename(plugin / "skills" / "demo-thing")
|
||||
_write(plugin / "README.md", "# demo\n\nSee the demo-thingamajig docs.\n")
|
||||
_check(
|
||||
ran,
|
||||
"skill satisfied only as a prefix of a longer word",
|
||||
any("does not name the skill 'demo-thing'" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
# The case a filename glob cannot see: a workflow ships under meta.name, so a README
|
||||
# citing only the filename leaves the invocable name unwritten anywhere. Both real
|
||||
# instances in this repo (git-cleanup, static-analysis) were exactly this shape.
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(
|
||||
plugin / "workflows" / "analyze.js",
|
||||
"export const meta = {\n name: 'demo-analysis',\n description: 'x',\n}\n",
|
||||
)
|
||||
_write(plugin / "README.md", "# demo\n\nSee `workflows/analyze.js`.\n")
|
||||
errors = _errors_for(root)
|
||||
_check(
|
||||
ran,
|
||||
"workflow named only by filename",
|
||||
any("does not name the workflow /demo:demo-analysis" in e for e in errors),
|
||||
)
|
||||
_write(plugin / "README.md", "# demo\n\nShips as `/demo:demo-analysis`.\n")
|
||||
_check(
|
||||
ran,
|
||||
"workflow named by meta.name is accepted",
|
||||
not any("does not name the workflow" in e for e in _errors_for(root)),
|
||||
)
|
||||
|
||||
|
||||
def _self_test_warnings(ran: list[str]) -> None:
|
||||
"""Each warning-level checker fires on a known-bad fixture, and does not block."""
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
@@ -1593,6 +1851,33 @@ def _self_test_warnings(ran: list[str]) -> None:
|
||||
not any("imaginary" in w for w in _warnings_for(root)),
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(plugin / "workflows" / "helper.js", "// no meta block here\nexport const x = 1\n")
|
||||
names = workflow_names(plugin)
|
||||
_check(ran, "helper .js without a meta block is not a workflow", names == [])
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_write(
|
||||
plugin / "workflows" / "w.js",
|
||||
"// name: 'from-a-comment'\nexport const meta = {\n name: 'real-name',\n}\n",
|
||||
)
|
||||
_check(
|
||||
ran,
|
||||
"workflow name read from meta, not an earlier comment",
|
||||
workflow_names(plugin) == [("w.js", "real-name")],
|
||||
)
|
||||
|
||||
# The per-kind floor: losing one discovery helper must go red, not hide in the total.
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
plugin = _build_demo(root)
|
||||
_, _, by_kind = validate_readme_names_components(plugin)
|
||||
_check(ran, "component scan reports a per-kind breakdown", by_kind.get("skill") == 1)
|
||||
|
||||
|
||||
def _self_test_guards(ran: list[str]) -> None:
|
||||
"""The anti-vacuity guards themselves."""
|
||||
@@ -1762,6 +2047,7 @@ def self_test() -> int:
|
||||
ran: list[str] = []
|
||||
try:
|
||||
_self_test_errors(ran)
|
||||
_self_test_readme_components(ran)
|
||||
_self_test_warnings(ran)
|
||||
_self_test_guards(ran)
|
||||
except AssertionError as exc:
|
||||
|
||||
@@ -279,6 +279,14 @@ Each of these fails the build. There is no value in checking any of it by hand:
|
||||
- `subagent_type` values are namespaced `<plugin>:<agent>` — a bare name is
|
||||
unregistered and the dispatch fails at runtime, whether it names this plugin's own
|
||||
agent, another plugin's, or nothing at all
|
||||
- A plugin's `README.md` names every skill, agent, command, and dynamic workflow it
|
||||
ships. A reader cannot invoke what is not written down, and the gap is worst where
|
||||
it is least guessable — a skill whose name is not the plugin name, or a workflow,
|
||||
which ships under its `meta.name` rather than its filename. Commands and workflows
|
||||
must appear in the `/<plugin>:<name>` form, since that is the only string a user can
|
||||
type; agents must appear as an identifier (backticked, an `agents/<name>` path, or
|
||||
namespaced), because a bare word in a sentence is not a dispatchable name; skills
|
||||
need only a delimited mention.
|
||||
- No hardcoded `/Users/…` or `/home/…` paths, in any `.md`, `.py`, `.json`, `.sh`,
|
||||
`.bats`, `.yml` or `.toml` file under `plugins/`. `*-shim.bats` is exempt because
|
||||
those fixtures need literal paths, and `/path/to` and `/home/vscode` are treated as
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "burpsuite-project-parser",
|
||||
"version": "1.0.1",
|
||||
"version": "1.0.2",
|
||||
"description": "Search and extract data from Burp Suite project files (.burp) for security analysis",
|
||||
"author": {
|
||||
"name": "Will Vandevanter",
|
||||
|
||||
@@ -37,6 +37,10 @@ This skill provides CLI access to Burp Suite project files through the burpsuite
|
||||
|
||||
## Usage
|
||||
|
||||
Inside Claude Code, run `/burpsuite-project-parser:burp-search` — it takes the same
|
||||
project path and flags as the script below.
|
||||
|
||||
|
||||
Base command:
|
||||
```bash
|
||||
scripts/burp-search.sh /path/to/project.burp [FLAGS]
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
---
|
||||
name: trailofbits:burp-search
|
||||
description: Searches Burp Suite project files for security analysis
|
||||
argument-hint: "<burp-file> [operation]"
|
||||
allowed-tools: Bash Read
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "c-review",
|
||||
"version": "2.0.0",
|
||||
"version": "2.0.1",
|
||||
"description": "Comprehensive C/C++ security code review, with coverage verified against a parse of the source",
|
||||
"author": {
|
||||
"name": "Paweł Płatek"
|
||||
|
||||
@@ -111,7 +111,7 @@ Two consequences worth knowing:
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `skills/c-review/SKILL.md` | The entry point. Collects the four parameters, resolves paths, makes one `Workflow` call, returns the report |
|
||||
| `workflows/c-review.js` | The orchestrator: the bug-class catalogue, every agent prompt, the phase sequence, and the deterministic duplicate merges |
|
||||
| `workflows/c-review.js` | The orchestrator, shipping as `/c-review:audit` — workflows are namespaced by `meta.name`, not the filename. The bug-class catalogue, every agent prompt, the phase sequence, and the deterministic duplicate merges |
|
||||
| `agents/c-review-worker.md` | The tool scope for review, sweep and dedup agents — see above |
|
||||
| `scripts/enumerate_units.py` | Parses the tree and cuts it into units, deciding which questions each unit owes and counting the spots each question is about. Everything downstream is partitioned from this |
|
||||
| `scripts/check_ledger.py` | The coverage check: re-derives those spots from the source and diffs them against what the agents wrote down |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "claude-in-chrome-troubleshooting",
|
||||
"version": "1.1.1",
|
||||
"version": "1.1.2",
|
||||
"description": "Diagnose and fix Claude in Chrome MCP extension connectivity issues",
|
||||
"author": {
|
||||
"name": "Dan Guido"
|
||||
|
||||
@@ -26,6 +26,10 @@ Diagnose and fix Claude in Chrome MCP extension connectivity issues.
|
||||
/plugin install trailofbits/skills/plugins/claude-in-chrome-troubleshooting
|
||||
```
|
||||
|
||||
The plugin ships one skill, `chrome-mcp-troubleshooting`. It triggers on its own when
|
||||
the symptoms above appear; invoke it directly with
|
||||
`/claude-in-chrome-troubleshooting:chrome-mcp-troubleshooting`.
|
||||
|
||||
## License
|
||||
|
||||
This work is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/).
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "constant-time-analysis",
|
||||
"version": "0.2.2",
|
||||
"version": "0.2.3",
|
||||
"description": "Detect compiler-induced timing side-channels in cryptographic code",
|
||||
"author": {
|
||||
"name": "Scott Arciszewski",
|
||||
|
||||
@@ -32,6 +32,16 @@ uv tool install .
|
||||
ct-analyzer crypto.c
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
The rest of this README documents `ct-analyzer`, the CLI. Inside Claude Code the plugin
|
||||
offers two more entry points:
|
||||
|
||||
| Component | Name | Purpose |
|
||||
|---|---|---|
|
||||
| Skill | `constant-time-analysis` | Triggers on its own when you write or review crypto code, or ask about a timing side channel. Invoke directly with `/constant-time-analysis:constant-time-analysis`. Adds the per-language guidance under `skills/constant-time-analysis/references/` and judgment about triaging what the analyzer reports |
|
||||
| Command | `/constant-time-analysis:ct-check` | One-shot run over a file — `<source-file> [--warnings] [--json] [--arch <arch>]` |
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Analysis
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
---
|
||||
name: trailofbits:ct-check
|
||||
description: Detects timing side-channels in cryptographic code
|
||||
argument-hint: "<source-file> [--warnings] [--json] [--arch <arch>]"
|
||||
allowed-tools: Bash Read Grep Glob
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "culture-index",
|
||||
"version": "1.1.2",
|
||||
"version": "1.1.3",
|
||||
"description": "Interprets Culture Index survey results for individuals and teams",
|
||||
"author": {
|
||||
"name": "Dan Guido"
|
||||
|
||||
@@ -29,6 +29,10 @@ This skill provides expert interpretation of Culture Index behavioral assessment
|
||||
/plugin install trailofbits/skills/plugins/culture-index
|
||||
```
|
||||
|
||||
The plugin ships one skill, `interpreting-culture-index`. It triggers on its own when
|
||||
you bring it a profile; invoke it directly with
|
||||
`/culture-index:interpreting-culture-index`.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Trait Colors
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "devcontainer-setup",
|
||||
"version": "0.2.1",
|
||||
"version": "0.2.2",
|
||||
"description": "Create pre-configured devcontainers with Claude Code and language-specific tooling",
|
||||
"author": {
|
||||
"name": "Alexis Challande",
|
||||
|
||||
@@ -16,6 +16,9 @@ Create pre-configured devcontainers with Claude Code and language-specific tooli
|
||||
|
||||
Tell Claude to "set up a devcontainer" or "add devcontainer support" in your project.
|
||||
|
||||
The plugin ships one skill, `devcontainer-setup`, which those phrases trigger. Invoke it
|
||||
directly with `/devcontainer-setup:devcontainer-setup`.
|
||||
|
||||
The skill will:
|
||||
1. Detect your project's language stack
|
||||
2. Generate `.devcontainer/` configuration files
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "differential-review",
|
||||
"version": "1.1.1",
|
||||
"version": "1.1.2",
|
||||
"description": "Security-focused differential review of code changes with git history analysis and blast radius estimation",
|
||||
"author": {
|
||||
"name": "Omar Inuwa",
|
||||
|
||||
@@ -28,19 +28,27 @@ This skill performs comprehensive security review of code changes:
|
||||
/plugin install trailofbits/skills/plugins/differential-review
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Name | Purpose |
|
||||
|---|---|---|
|
||||
| Skill | `differential-review` | The review itself — Pre-Analysis plus Phases 0–6, described below |
|
||||
| Command | `/differential-review:diff-review` | Entry point taking `<pr-url\|commit-sha\|diff-path> [--baseline <ref>]` |
|
||||
| Agent | `differential-review:adversarial-modeler` | Phase 5 only. The skill delegates to it for HIGH RISK changes to build concrete exploit paths and rate exploitability |
|
||||
|
||||
## Documentation Structure
|
||||
|
||||
This skill uses a **modular documentation architecture** for token efficiency and progressive disclosure:
|
||||
|
||||
### Core Entry Point
|
||||
- **[SKILL.md](skills/differential-review/SKILL.md)** - Main entry point (217 lines)
|
||||
- **[SKILL.md](skills/differential-review/SKILL.md)** - Main entry point (224 lines)
|
||||
- Quick reference tables for triage
|
||||
- Decision tree routing to detailed docs
|
||||
- Quality checklist and red flags
|
||||
- Integration with other skills
|
||||
|
||||
### Supporting Documentation
|
||||
- **[methodology.md](skills/differential-review/methodology.md)** - Detailed phase-by-phase workflow (~200 lines)
|
||||
- **[methodology.md](skills/differential-review/methodology.md)** - Detailed phase-by-phase workflow (234 lines)
|
||||
- Pre-Analysis: Baseline context building
|
||||
- Phase 0: Intake & Triage
|
||||
- Phase 1: Changed Code Analysis
|
||||
@@ -48,19 +56,19 @@ This skill uses a **modular documentation architecture** for token efficiency an
|
||||
- Phase 3: Blast Radius Analysis
|
||||
- Phase 4: Deep Context Analysis
|
||||
|
||||
- **[adversarial.md](skills/differential-review/adversarial.md)** - Attacker modeling and exploit scenarios (~150 lines)
|
||||
- **[adversarial.md](skills/differential-review/adversarial.md)** - Attacker modeling and exploit scenarios (203 lines)
|
||||
- Phase 5: Adversarial Vulnerability Analysis
|
||||
- Attacker model definition (WHO/ACCESS/INTERFACE)
|
||||
- Exploitability rating framework
|
||||
- Complete exploit scenario templates
|
||||
|
||||
- **[reporting.md](skills/differential-review/reporting.md)** - Report structure and formatting (~120 lines)
|
||||
- **[reporting.md](skills/differential-review/reporting.md)** - Report structure and formatting (369 lines)
|
||||
- Phase 6: Report Generation
|
||||
- 9-section report template
|
||||
- Formatting guidelines and conventions
|
||||
- File naming and notification templates
|
||||
|
||||
- **[patterns.md](skills/differential-review/patterns.md)** - Common vulnerability patterns (~80 lines)
|
||||
- **[patterns.md](skills/differential-review/patterns.md)** - Common vulnerability patterns (300 lines)
|
||||
- Security regressions detection
|
||||
- Reentrancy, access control, overflow patterns
|
||||
- Quick detection bash commands
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
---
|
||||
name: trailofbits:diff-review
|
||||
description: Performs security-focused differential review of code changes
|
||||
argument-hint: "<pr-url|commit-sha|diff-path> [--baseline <ref>]"
|
||||
allowed-tools: Read Write Grep Glob Bash
|
||||
|
||||
@@ -1,10 +1,6 @@
|
||||
---
|
||||
name: differential-review
|
||||
description: >
|
||||
Performs security-focused differential review of code changes (PRs, commits, diffs).
|
||||
Adapts analysis depth to codebase size, uses git history for context, calculates
|
||||
blast radius, checks test coverage, and generates comprehensive markdown reports.
|
||||
Automatically detects and prevents security regressions.
|
||||
description: "Performs security-focused differential review of code changes. Adapts analysis depth to codebase size, uses git blame for context, calculates blast radius by counting callers, checks test coverage of modified code, and generates a markdown report. Use when reviewing a PR, commit, or diff for security vulnerabilities, checking whether a change re-introduces a previously fixed bug, asking what else a change could break, or finding which modified code has no test covering it."
|
||||
allowed-tools: Read Write Grep Glob Bash
|
||||
---
|
||||
|
||||
@@ -78,7 +74,7 @@ Phase 3: Blast Radius → Phase 4: Deep Context → Phase 5: Adversarial → Pha
|
||||
├─ Analyzing HIGH RISK change?
|
||||
│ ├─ Read: adversarial.md
|
||||
│ │ (Phase 5: Attacker modeling, exploit scenarios, exploitability rating)
|
||||
│ └─ Or delegate to: adversarial-modeler agent
|
||||
│ └─ Or delegate to: differential-review:adversarial-modeler agent
|
||||
│ (Autonomous attacker modeling with concrete exploit scenarios)
|
||||
│
|
||||
├─ Writing the final report?
|
||||
@@ -97,12 +93,13 @@ Phase 3: Blast Radius → Phase 4: Deep Context → Phase 5: Adversarial → Pha
|
||||
|
||||
## Agents
|
||||
|
||||
**`adversarial-modeler`** — Models attacker perspectives and builds exploit
|
||||
scenarios for HIGH RISK code changes. Follows the 5-step adversarial
|
||||
methodology (attacker model, attack vectors, exploitability rating, exploit
|
||||
scenario, baseline cross-reference) and produces structured vulnerability
|
||||
**`differential-review:adversarial-modeler`** — Models attacker perspectives and
|
||||
builds exploit scenarios for HIGH RISK code changes. Follows the 5-step
|
||||
adversarial methodology (attacker model, attack vectors, exploitability rating,
|
||||
exploit scenario, baseline cross-reference) and produces structured vulnerability
|
||||
reports. Delegate to this agent when Phase 5 analysis is needed on high-risk
|
||||
changes.
|
||||
changes, passing that full namespaced name as `subagent_type` — a bare
|
||||
`adversarial-modeler` is unregistered and the dispatch fails at runtime.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "entry-point-analyzer",
|
||||
"version": "1.0.1",
|
||||
"version": "1.0.2",
|
||||
"description": "Analyzes smart contract codebases to identify state-changing entry points for security auditing. Detects externally callable functions that modify state, categorizes them by access level, and generates structured audit reports.",
|
||||
"author": {
|
||||
"name": "Nicolas Donboly",
|
||||
|
||||
@@ -43,6 +43,10 @@ Trigger the skill with requests like:
|
||||
- "List audit flows for src/core/"
|
||||
- "What privileged operations exist in this project?"
|
||||
|
||||
Or invoke either entry point directly:
|
||||
- Skill — `/entry-point-analyzer:entry-point-analyzer`
|
||||
- Command — `/entry-point-analyzer:entry-points [directory-path]`
|
||||
|
||||
## Directory Filtering
|
||||
|
||||
Specify a subdirectory to limit scope:
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
---
|
||||
name: trailofbits:entry-points
|
||||
description: Identifies state-changing entry points in smart contracts
|
||||
argument-hint: "[directory-path]"
|
||||
allowed-tools: Read Grep Glob Bash
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "git-cleanup",
|
||||
"version": "2.3.0",
|
||||
"version": "2.3.1",
|
||||
"description": "Safely analyzes and cleans up local git branches and worktrees by categorizing them as merged, squash-merged, superseded, or active work.",
|
||||
"author": {
|
||||
"name": "Henrik Brodin",
|
||||
|
||||
@@ -53,7 +53,7 @@ claude plugins:add trailofbits/skills/git-cleanup
|
||||
| Path | Role |
|
||||
|------|------|
|
||||
| `commands/git-cleanup.md` | The `/git-cleanup` entry point: gates, confirmations, and the deletions |
|
||||
| `workflows/analyze-branches.js` | The dynamic workflow. Read-only — it never deletes |
|
||||
| `workflows/analyze-branches.js` | The dynamic workflow, shipping as `/git-cleanup:git-cleanup-analysis` — workflows are namespaced by `meta.name`, not the filename. Read-only — it never deletes |
|
||||
| `references/merge-evidence.md` | What counts as proof a branch is merged, read by the agents and the fallback path |
|
||||
| `tests/analyze-branches.test.mjs` | Stubs every agent and asserts the triage, clustering, and failure handling |
|
||||
| `evals/` | End-to-end evals: does the model produce a correct gate-1 analysis on a real repository? |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "insecure-defaults",
|
||||
"version": "2.0.0",
|
||||
"version": "2.0.1",
|
||||
"description": "Detects insecure default configurations including hardcoded credentials, fallback secrets, weak authentication defaults, and dangerous values in production",
|
||||
"author": {
|
||||
"name": "Trail of Bits",
|
||||
|
||||
@@ -40,7 +40,7 @@ Each category is three files that must agree:
|
||||
|
||||
| | |
|
||||
| ---------------------- | -------------------------------------------------------------------- |
|
||||
| `workflows/audit.js` | an `{ id, title }` row, all the script knows |
|
||||
| `workflows/audit.js` | an `{ id, title }` row, all the script knows. Ships as `/insecure-defaults:audit-pipeline` |
|
||||
| `references/<id>.json` | title + seed patterns |
|
||||
| `references/<id>.md` | **Report when** / **Skip when**, plus worked vulnerable/secure pairs |
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "let-fate-decide",
|
||||
"version": "1.2.2",
|
||||
"version": "1.2.3",
|
||||
"description": "Draws the 12 Houses of the Zodiac Tarot spread using cryptographic randomness to add 100+ bits of entropy to vague or underspecified planning. Interprets the spread to guide next steps. Use when feeling lucky, invoking heart-of-the-cards energy, or when prompts are ambiguous.",
|
||||
"author": {
|
||||
"name": "Scott Arciszewski",
|
||||
|
||||
@@ -24,6 +24,10 @@ interprets the spread and uses the reading to inform its approach.
|
||||
|
||||
## How It Works
|
||||
|
||||
The spread is drawn by the `draw` agent (`agents/draw.md`), dispatched as
|
||||
`let-fate-decide:draw`.
|
||||
|
||||
|
||||
1. A Python script uses `secrets.randbelow()` to perform Fisher-Yates shuffles
|
||||
2. A Major Arcana deck and Minor Arcana deck are shuffled separately
|
||||
3. 12 houses are dealt, each with 1 Major Arcana and 2 Minor Arcana cards
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "mutation-testing",
|
||||
"version": "1.0.1",
|
||||
"version": "1.0.2",
|
||||
"description": "Configures mewt or muton mutation testing campaigns — scopes targets, tunes timeouts, and optimizes long-running runs. Use when the user mentions mewt, muton, mutation testing, or wants to configure or optimize a mutation testing campaign.",
|
||||
"author": {
|
||||
"name": "Trail of Bits",
|
||||
|
||||
@@ -4,6 +4,9 @@ Helps configure [mewt](https://github.com/trailofbits/mewt) or [muton](https://g
|
||||
|
||||
> **Note**: muton and mewt share identical interfaces but target different languages — mewt for general-purpose languages, muton for TON smart contracts (Tact, Tolk, FunC). All commands and configuration patterns in this plugin apply to both tools. File names change accordingly: `mewt.toml` → `muton.toml`, `mewt.sqlite` → `muton.sqlite`.
|
||||
|
||||
The plugin ships one skill, `mutation-testing`, which the triggers below fire. Invoke it
|
||||
directly with `/mutation-testing:mutation-testing`.
|
||||
|
||||
## What It Does
|
||||
|
||||
Walks through a 5-phase configuration workflow:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "semgrep-rule-creator",
|
||||
"version": "1.2.3",
|
||||
"version": "1.2.4",
|
||||
"description": "Create custom Semgrep rules for detecting bug patterns and security vulnerabilities",
|
||||
"author": {
|
||||
"name": "Maciej Domanski"
|
||||
|
||||
@@ -36,6 +36,11 @@ Use this skill when you need to:
|
||||
/plugin install trailofbits/skills/plugins/semgrep-rule-creator
|
||||
```
|
||||
|
||||
Then run `/semgrep-rule-creator:semgrep-rule` to walk through building a rule.
|
||||
|
||||
```
|
||||
```
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `semgrep-rule-variant-creator` - Port existing Semgrep rules to new target languages
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
---
|
||||
name: trailofbits:semgrep-rule
|
||||
description: Creates Semgrep rules with test-first methodology
|
||||
argument-hint: "(uses conversation context for detection pattern)"
|
||||
allowed-tools: Bash Read Write Edit Glob Grep WebFetch
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "static-analysis",
|
||||
"version": "1.4.0",
|
||||
"version": "1.4.1",
|
||||
"description": "Static analysis toolkit with CodeQL, Semgrep, and SARIF parsing for security vulnerability detection",
|
||||
"author": {
|
||||
"name": "Axel Mierczuk & Paweł Płatek"
|
||||
|
||||
@@ -70,6 +70,7 @@ the same `references/`, so a ruleset added to `rulesets.md` reaches both.
|
||||
| Workflow | Purpose |
|
||||
|----------|---------|
|
||||
| `workflows/semgrep-scan.js` | Ships as `/static-analysis:semgrep-scan`. Four phases: Detect, Select, Scan, Report |
|
||||
| `workflows/codeql-build.js` | Ships as `/static-analysis:codeql-build`. Three phases: Detect, Build, Assess. Walks the build-method ladder — autobuild, custom command, multi-step, no-build — diagnosing and retrying at each rung, then enforces the quality gate |
|
||||
|
||||
## Scripts Included
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "trailmark",
|
||||
"version": "0.11.1",
|
||||
"version": "0.11.2",
|
||||
"description": "Builds multi-language source and binary code graphs for security analysis: call graphs, attack surface mapping, blast radius, taint propagation, complexity hotspots, entry point enumeration, proxy/unresolved-call tracking, type/reference analysis, and structural diffs. Creates bounded, graph-informed source packets for delegating focused work to constrained subagents. Generates Mermaid diagrams, runs graph-informed mutation testing triage (genotoxic), generates mutation-driven test vectors (vector-forge), extracts crypto protocol message flows, converts Mermaid diagrams to ProVerif models, projects SARIF/weAudit/binary findings onto code graphs, triages single findings with graph evidence, gates branch diffs for structural review regressions, and expands seed findings into variant-neighborhood candidates. Use when analyzing call paths, slicing source context for smaller models, mapping attack surface, visualizing code architecture, triaging survived mutants, generating cryptographic test vectors, diagramming crypto protocols, formally verifying protocols, augmenting audits with static analysis findings, deciding whether one candidate issue is reachable, reviewing graph-level PR risk, or seeding variant analysis.",
|
||||
"author": {
|
||||
"name": "Scott Arciszewski",
|
||||
|
||||
@@ -75,7 +75,8 @@ trailmark/
|
||||
├── .claude-plugin/
|
||||
│ └── plugin.json
|
||||
├── agents/
|
||||
│ └── code-slice-worker.md # Repository-tool-free bounded worker
|
||||
│ └── code-slice-worker.md # Bounded worker, no repository tools.
|
||||
│ # Dispatch as `trailmark:code-slice-worker`
|
||||
├── README.md
|
||||
└── skills/
|
||||
├── trailmark/ # Core graph querying
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "zeroize-audit",
|
||||
"version": "0.3.0",
|
||||
"version": "0.3.1",
|
||||
"description": "Detects missing or compiler-optimized zeroization of sensitive data with assembly and control-flow analysis",
|
||||
"author": {
|
||||
"name": "Trail of Bits",
|
||||
|
||||
@@ -91,10 +91,10 @@ bear -- make -j$(nproc)
|
||||
|
||||
## Agent Architecture
|
||||
|
||||
The analysis pipeline uses 10 agents across 8 phases, enabling parallel source analysis (C/C++ and Rust simultaneously), per-TU compiler analysis, mandatory PoC validation with verification, and protection against context pressure:
|
||||
The analysis pipeline uses 11 agents across 9 phases (0-8), enabling parallel source analysis (C/C++ and Rust simultaneously), per-TU compiler analysis, mandatory PoC validation with verification, and protection against context pressure:
|
||||
|
||||
```
|
||||
Phase 0: Orchestrator — Preflight + config + create workdir + enumerate TUs
|
||||
Phase 0: 0-preflight agent — Preflight + config + create workdir + enumerate TUs
|
||||
Phase 1: Wave 1: 1-mcp-resolver (skip if mcp_mode=off OR language_mode=rust)
|
||||
Wave 2a: 2-source-analyzer (C/C++ only; skip if no compile_db) ─┐ parallel
|
||||
Wave 2b: 2b-rust-source-analyzer (Rust only; skip if no cargo_manifest) ─┘
|
||||
@@ -114,6 +114,7 @@ Phase 8: Orchestrator — Return final-report.md
|
||||
|
||||
| Agent | Phase | Purpose | Output Directory |
|
||||
|---|---|---|---|
|
||||
| `0-preflight` | 0 | Validate prerequisites, merge config, enumerate TUs, create the run directory. Gates everything after it — if a critical check fails, the run stops here | run root |
|
||||
| `1-mcp-resolver` | 1, Wave 1 | Resolve symbols/types via Serena MCP (C/C++ only) | `mcp-evidence/` |
|
||||
| `2-source-analyzer` | 1, Wave 2a | Sensitive objects, wipes, data-flow/heap (C/C++) | `source-analysis/` |
|
||||
| `2b-rust-source-analyzer` | 1, Wave 2b | Rustdoc JSON trait analysis + dangerous API grep | `source-analysis/` |
|
||||
@@ -121,6 +122,8 @@ Phase 8: Orchestrator — Return final-report.md
|
||||
| `3b-rust-compiler-analyzer` | 2, Wave 3R | Crate-level MIR + LLVM IR analysis (Rust) | `rust-compiler-analysis/` |
|
||||
| `4-report-assembler` | 3+6, Wave 4+6 | Collect findings, confidence gating; merge PoC results (invoked twice: interim + final) | `report/` |
|
||||
| `5-poc-generator` | 4, Wave 5 | Generate proof-of-concept programs (C/C++ findings only) | `poc/` |
|
||||
| `5b-poc-validator` | 5, Step 5a | Compile and run every PoC | `poc/` |
|
||||
| `5c-poc-verifier` | 5, Step 5b | Check each PoC actually proves the finding it claims, by reading the PoC, the finding, and the original source | `poc/` |
|
||||
| `6-test-generator` | 7, Wave 7 | Generate runtime validation test harnesses (optional) | `tests/` |
|
||||
|
||||
Agents write persistent finding files to a shared working directory (`/tmp/zeroize-audit-{run_id}/`) with namespaced IDs to prevent collisions during parallel execution.
|
||||
|
||||
Reference in New Issue
Block a user