Files
Dan Guido 9e06dc67a3 Make every documented command runnable under our own python shims (#258)
* Make every documented command runnable under our own python shims

The modern-python plugin ships PATH shims that refuse `python <script>`,
`pip install`, `python -m pip` and `uv pip install`. Twelve other plugins
in this marketplace issued exactly those forms, so installing our own
plugin broke our own skills — and CI was green throughout.

The worst case was not theoretical. c-review and rust-review both call
their Phase 4 planner as `python3 "${PLUGIN_ROOT}/scripts/build_run_plan.py"`,
so with the shim installed every run died before spawning a worker.
Verified both directions: the new form exits 0 with the shim on PATH, the
old form exits 1.

Phase 1's reading pass named 16 skills. A mechanical sweep found 96
candidate lines across 44 files, and scanning shell scripts as well as
markdown found 10 more the docs sweep had missed. That gap is the reason
the check below exists.

The fix is not one substitution. Four classes needed different treatment:

- Our own scripts become `uv run --no-project <script>`. Not bare `uv run`,
  because these execute inside the *target* repo, which may be a Python
  project that cannot resolve; verified against a broken pyproject.toml and
  against validate_artifacts.py's sibling import of generate_sarif.
- Package installs become `uv add` for a dependency, `uv tool install` for a
  CLI, `uv sync` for a project's own editable install.
- Third-party CLIs we merely document — OSS-Fuzz's infra/helper.py, yarGen —
  become `uv run --no-project python <script>`, which keeps upstream's exact
  semantics rather than handing their script an environment we manage.
- atheris's instrumented build keeps its source build, as
  `uv add --no-binary-package cbor2`. Dropping that flag would silently
  produce an uninstrumented fuzzer, which is worse than a visible failure.
  Its prose was updated to name the flag it now uses.

Two factual corrections fell out. `pip install caracal` was wrong twice
over: caracal is a Rust tool (Cargo.toml at its root), so it is now
upstream's own `cargo install --git`, not a uv equivalent that would fetch
an unrelated PyPI package. And `pip install uv` cannot bootstrap uv under
a shim that intercepts pip, so culture-index now points at the official
installer.

Thirteen lines stay as they are, each deliberately: Dockerfile `RUN` lines
and oss-fuzz's build.sh run in containers where our shims are absent;
codeql's pip calls install the *analysed* project's dependencies, and that
project is arbitrary; trailmark's dispatch skills must keep saying "Do NOT
run `pip install`"; and modern-python documents what it intercepts.

`make shell-suites` passes again as a result — exit 0 with the 1.6.0 shim,
where AGENTS.md previously recorded it as broken by variant-analysis.

The guardrail: check_python_invocations scans 698 markdown and shell files
and fails on the four refused forms, with structural exemptions for
dockerfile fences and an `allow-legacy-python: <reason>` marker that scopes
to its code block. Eleven self-test fixtures cover it, four asserting it
fires and seven asserting it stays quiet on the compliant forms. It was
mutation-tested in both languages, and it caught its own worst bug during
development: unanchored patterns first flagged `uv run --no-project python
fuzz.py`, the very form the advice recommends. Self-test goes 45 -> 56.

* Review pass: fix the atheris flow, drop a stray exemption, trim comments

Three corrections from reviewing the branch diff:

- atheris's install now opens with `uv init --bare`, without which the
  documented `uv add atheris` errors in a bare harness directory. The old
  pip form assumed an activated venv, so setup was always implicit; now
  it is one explicit line.
- ossfuzz carried an allow-legacy-python marker on a C++ build block that
  contains no python at all — yesterday's insertion matched the first of
  three "Build in build.sh" headings instead of the python one. The
  exemption now sits only on the block that needs it.
- The anti-vacuity message said "read no markdown" for a scan that also
  covers shell scripts.

The rest is weight: the new check's comment blocks, the hardcoded-path
constants' commentary, the AGENTS.md bullets and the three exemption
markers all said the same things at two to three times the length. Each
keeps its one-line why; the narratives are gone. No behavioural change —
self-test still passes 56 assertions and the full scan is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Address the review: fix where packages land, widen the check to match the shim

The review's core insight was right twice over. Several substitutions had
changed WHERE a package lands, breaking the documented next step, and the
checker enforced a narrower invariant than the shim it exists to mirror.

Where packages land:

- trailmark is imported as a library from five skills, and a `uv tool
  install` environment is not importable — the retry loop at
  trailmark/SKILL.md:47-51 would have spun forever on the exact error it
  names. The CLI install stays `uv tool install`; the import snippets now
  run under `uv run --with trailmark python -`.
- `uv add` writes to the manifest of whatever project you are standing
  in, which for sarif-parsing is the audited repo. Its scripting rows,
  ijson comment and jsonschema example now use `uv run --with <pkg>`,
  which leaves no trace. atheris keeps `uv add` deliberately: the fuzzing
  harness is the user's own project, made explicit by `uv init --bare`.
- `uv sync` leaves ct-analyzer in .venv/bin, so the README's very next
  line failed with command not found. Now `uv tool install .`, verified
  end to end: the console script lands on PATH and --help runs.
- yarGen needs pefile/lxml/yara-python, which `--no-project` had detached;
  now `uv run --with-requirements requirements.txt`.
- The cbor2 source-build preference now persists via
  `no-binary-package = ["cbor2"]` under [tool.uv] (field verified against
  uv's accepted-settings list), so a later `uv sync` cannot silently swap
  in an uninstrumented wheel.

The checker, widened to the shim's actual behaviour:

- `python3 --version` and `python3 -u foo.py` are refused by the shim but
  passed the old patterns; one live instance (constant-time-analysis
  README) proved it. Both forms are now caught.
- Every `uv pip` subcommand is refused, not just install; `-t` joins the
  allowed tool-managed flags.
- .py files are scanned too: usage strings and error messages told users
  to run refused commands from ten scripts, including the --help of the
  very planner this PR fixed. All rewritten.
- The evals/tests exemption now tests path parts relative to plugins/, so
  a checkout under a directory named tests no longer exempts every file.
- An allow-marker's scope ends at a blank line as well as a fence, so one
  marker cannot blanket a whole file; quality-assessment.md gains the
  second marker that scoping made necessary.

Also from the review: zeroize's preflight gets `which python3` back (a
helper script still needs the binary; the shim never required removing
it), the Makefile's shell-suites note no longer describes an interception
that is gone, and the cairo CI example warns that it rebuilds caracal
from source each run.

Self-test 56 -> 63; every new pattern and exemption is fixture-covered
and was mutation-probed against the real tree. Full scan: 0 findings over
773 files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Address the second review: prerequisite probe, checker parity, package placement

The review's P2 was a regression this PR introduced for a population the
first fix ignored: c-review and rust-review now require uv, and a box
with python3 but no uv would die at Phase 4 exactly the way shimmed boxes
died before. Phase 1 (Prerequisites) in both skills now probes
`command -v uv` and aborts with install guidance. zeroize-audit's
preflight already checked uv. The four converted shell suites gain the
same guard with a clear message instead of a bare 127 mid-run.

Checker parity with the shims, second pass:

- pipx and the non-install pip subcommands are refused by catch-all shim
  arms and passed the checker; both get named-subcommand patterns.
- A script named by variable or path (`python3 "$MERGE"`) has no `.py`
  token; a new pattern covers it and immediately caught one live
  instance — a codeql test stub that fakes uv itself, now carrying an
  allow-marker with its reason.
- finditer everywhere: a compliant `uv run` earlier on a line no longer
  masks a refused command later on it, which was exactly the table-cell
  case the unanchored design exists for.
- Prohibition phrases now test the text BEFORE the match, so
  "Use `pip install semgrep` instead of the tarball" is flagged while
  "Do NOT run `pip install`" stays exempt.
- The uv-pip allowance matches whole flags after the command, so
  `--target-dir` no longer counts as `--target` and a trailing `-t /tmp`
  does; `uv pip` precedes `pip` in the pattern order so its lines get
  the right advice; a pip match directly after `uv ` defers to the
  uv-pip verdict instead of double-reporting.

Package placement, continued from the same insight as round one:

- yarGen regains --no-project alongside --with-requirements, plus a cd
  into the checkout so requirements.txt resolves where it lives.
- sarif-parsing's jsonschema example no longer names a script that does
  not exist, and the table's run-forms show a concrete script.py.
- culture-index's two messages now agree and name the actual remedy
  (`uv run --project` on the scripts directory) instead of re-adding a
  dependency its pyproject already declares.
- merge_sarif's usage line gains --no-project; the generator plugin's
  install section stops prescribing a venv its own runner never uses.
- generate_poc declared requires-python >=3.9 while using `str | None`
  in a signature, a TypeError on 3.9 that uv's interpreter selection
  made reachable; now >=3.10.
- The GitLab CI example exports ~/.local/bin onto PATH, without which
  `uv tool install` warns and the next line dies command-not-found.

Self-test 63 -> 71; the masking, prohibition-direction, flag-position
and pipx cases are all fixtures, and each new pattern was probed live
against the tree (plant, error, remove, clean — 0 findings over 773
files).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Address the third review: importable trailmark, honest probes, sturdier scan

The P2 was the residue of round two's own fix, applied to the siblings
but not the flagship: trailmark/SKILL.md told the model to cure an import
error with `uv tool install`, which cannot cure it — a tool env is not
importable — while forbidding every fallback. The install block now says
what each remedy is for: `uv tool install` for the CLI, `uv run --with
trailmark python -` for the snippets, and the other five library-first
docs carry the same one-line annotation next to their install command.

Empirically settled rather than taken from the review: `uv run python3
<script>` works fine under the shims — uv prepends its environment's bin
directory, so python3 resolves to a real interpreter, not the shim. The
review's claim to the contrary would have meant rewriting the Makefile
and a bats suite; a two-minute transcript said no. Also declined: a
zeroize uv-prerequisite (its preflight already lists uv and uvx; the
C/C++ `which` line now names uv too).

Real and fixed:

- ct-analyzer's availability probe ran `python3 --version` by subprocess
  — the one refused form — so under the shims it reported "Python is not
  available" on machines where it plainly is. It now probes
  sys.executable, the interpreter the analyzer itself runs under.
  Verified under the shim: probe returns True.
- The flag step-over in both script patterns handles long and
  value-taking flags (`python3 -W ignore harness.py`, `--verbose
  tool.py`), matching the shim's two-slot consumption.
- A bare `allow-legacy-python:` with no reason no longer exempts
  anything; the reason the docs demand is now enforced.
- `uv run {baseDir}/...` gets --no-project at the ten semgrep and
  culture-index call sites that round two missed, and the culture-index
  remediation strings now name that same runnable command instead of a
  --project mechanism nothing uses.
- pip gains cache/config; the pattern comment now says the subcommand
  list is deliberately a subset.
- Both filesystem scans skip .venv/node_modules-style directories, after
  a stray local .venv (left by this session's own uv probe, and invisible
  to CI) turned the path scan red.

Smaller review items: the uv-probe prose says "Phase 4 onward" rather
than a wrong phase range, run_fixtures' comment stops claiming PEP 723
headers its stdlib-only helpers do not have, the yarGen one-liners say
to run from the checkout, `uv tool install` sites note or export the
tool bin dir the way a fresh container needs, and sarif-parsing's table
column says Install / run and stops naming a file that does not exist.

Self-test 71 -> 74. Full scan: 0 findings over 773 files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 13:39:07 -04:00
..
2026-06-30 11:14:46 -04:00
2026-06-30 11:14:46 -04:00

rust-review

Rust security code review plugin. Bug-class coverage comes from empirical bug-shape research across 245 memory-corruption, 177 unsound safe-API, 150 denial-of-service, and 60 thread-safety advisories in the RustSec Advisory Database (1,078 entries) and audits. Orchestration matches c-review.

Usage

Invoke with /rust-review:rust-review. The skill will prompt for:

  • Threat model (REMOTE / LOCAL_UNPRIVILEGED / BOTH)
  • Worker model (haiku / sonnet / opus)
  • Severity filter (all / medium / high)
  • Scope subpath (optional — defaults to whole repo)

Findings + SARIF are written to $(pwd)/.rust-review-results/<iso-timestamp>/.

Overview

Inputs (AskUserQuestion): threat model, scope subpath (optional), worker model, severity filter.

From these inputs the orchestrator detects Rust capability flags (has_unsafe, has_ffi, has_concurrency, has_async, has_packed_repr, has_fs_io) over the scope and selects clusters from prompts/clusters/manifest.json. Each cluster groups related bug classes anchored on a shared mental model and runs as one parallel worker.

The planner caps each non-consolidated worker at four passes, splitting larger clusters into -1/-2/… chunks; output-heavy clusters can declare a smaller max_passes_per_worker in the manifest (today recursion-dos runs one pass per worker). Consolidated clusters (unsafe-boundary, concurrency-locking) are never chunked — one worker owns the whole cluster so its shared Phase-A inventory is built once and grounds every phase.

Always-on clusters:

  • unsafe-boundary (consolidated) — Unsafe Reachability Analysis (URAPI), transmute misuse, pointer-cast hazards via as (PTRCAST), raw-pointer arithmetic, #[repr(C)] layout, enum discriminant and niche validity (ENUMUB), // SAFETY: documentation rules, debug_assert!-guarded safety invariants.
  • panic-dos — resource exhaustion DoS (RESEXHAUST, P0), unwrap/expect on untrusted input, arithmetic overflow, reachable unreachable!/assert!, vector OOB indexing, non-char-boundary str slicing panics (STRSLICE), reachable RefCell double-borrow panics (REFCELLPANIC).
  • recursion-dos — stack-overflow aborts (uncatchable, distinct from panics) on recursive types: unbounded deserialization depth (serde_yaml/toml/ron/custom Deserialize), recursive Display/Debug/Serialize on attacker-shaped values, implicit Drop of Box<Self>-style chains.
  • error-handling — discarded Results, panics inside Drop, lossy From/Into and as casts, lossy UTF-8 / OS-string / path conversions (LOSSYSTR), unflushed BufWriter swallowing write errors (BUFFLUSH).
  • logic-correctnessOrd/Eq/Hash invariant violations, hostile generic trait impls, closure-panic across unsafe scaffolding, NaN/Inf edge cases, partial-match/case string comparisons (STRCMP), serialize_struct field-count mismatches (SERFIELDS), nondeterminism in replicated state (NONDET), in-collection key mutation (KEYMUT). The hostile-trait (TRAITADV) and closure-panic (CLOSUREPANIC) passes require has_unsafe.
  • static-hygiene — Cargo lint config, MSRV, deprecated APIs (mem::uninitialized).
  • resource-handling — raw file-descriptor double-close and leak (RAWFD), Drop-skipping cleanup via process::exit/mem::forget (DROPSKIP).
  • info-disclosure — pointer/address exposure that defeats ASLR (PTREXPOSE).

Conditional clusters:

  • memory-safety (has_unsafe) — UAF via dangled raw pointer, double-free via ptr::read, invalid-free via assignment-to-uninit, uninitialized-read via premature assume_init, Vec::set_len without slot init (SETLEN), buffer overflow via safe→unsafe index propagation, union variant misread, panic-unsafe custom container drop (PANICUNWIND). The whole cluster is gated on has_unsafe (every memory-safety bug class requires unsafe), so it is omitted entirely for pure safe-Rust crates.
  • concurrency-locking (has_concurrency, consolidated) — MutexGuard double-lock from lexical scope, ABBA ordering, Condvar wait without notifier, channel starvation, Once::call_once reentrancy, signal-handler / callback reentrancy.
  • concurrency-data-race (has_concurrency) — non-atomic atomic sequences, unsafe impl Sync over interior mutability, missing Send/Sync bounds, cross-process shared-memory races, unsynchronized static mut (STATICMUT). The unsafe-sync-impl (UNSAFESYNC) and static-mut (STATICMUT) passes require has_unsafe.
  • ffi-cross-language (has_ffi) — CString::as_ptr dangling, ABI mismatch, #[repr(C)] padding leak, opaque-pointer ownership confusion, FFI-owned-memory drop mismatches, Rust closures across extern "C" without catch_unwind, dyn Trait fat pointers crossing FFI.
  • layout-safety (has_packed_repr) — unaligned references to #[repr(packed)] / wire-format struct fields (PACKEDREF).
  • input-os-safety (has_fs_io) — PathBuf::join path traversal (PATHJOIN), filesystem TOCTOU (TOCTOU).
  • async-runtime (has_async) — blocking calls in async, cancellation-unsafe .await sequences, tokio::select! branch bias.

Same orchestration as c-review: workers spawn foreground (one message per wave of ≤16 workers, after an optional cache primer), write markdown-with-YAML-frontmatter finding files, then a dedup-judge merges duplicates, then an fp-judge assigns fp_verdict / severity / attack_vector / exploitability. A report safety net then runs: SARIF is regenerated unconditionally, and the orchestrator writes REPORT.md itself if the fp-judge failed to.

Architecture

coordinator: write context.md → build_run_plan.py → TaskCreate × M
          → spawn primer (foreground) → spawn M workers (parallel)
          → classify Phase-7 outcomes + write findings-index.txt
          → dedup-judge → fp-judge → report safety net (SARIF + REPORT.md) → return REPORT.md
Subagent type Purpose Tool set
rust-review:rust-review-worker Run assigned cluster, write findings Read, Write, Edit, Bash
rust-review:rust-review-dedup-judge Merge duplicates (runs first) Read, Write, Edit, Glob
rust-review:rust-review-fp-judge FP + severity + final reports (runs second) Read, Write, Edit, Bash

In current Claude Code an agent granted Bash is not also granted the dedicated Glob/Grep tools (the harness expects find/grep/rg via Bash). So the worker and fp-judge search and resolve paths with Read/Bash, running the ripgrep-syntax prompt seeds through rg; only the dedup-judge — which holds no Bash — uses Glob.

Output directory layout

Default: $(pwd)/.rust-review-results/<iso-timestamp>/. Contains:

  • context.md — resolved threat model, severity filter, scope, capability flags, Cargo manifest status
  • plan.json — selected clusters + rendered worker spawn prompts (one per parallel worker)
  • worker-prompts/ — verbatim spawn prompts, one per worker (+ optional cache-primer.txt)
  • findings/ — one markdown file per finding (<PREFIX>-NNN.md with YAML frontmatter)
  • findings-index.d/ — per-worker shards listing finding paths (survive an orchestrator crash)
  • findings-index.txt — canonical sorted list of every finding file on disk (reconciled against the shards)
  • run-summary.md — worker outcome table, retry/abort state, judge status
  • dedup-summary.md — Tier 13 merge + Tier 4 related summary
  • fp-summary.md — verdict counts and per-primary verdict table
  • REPORT.md — human-readable final report grouped by severity, filtered per severity_filter
  • REPORT.sarif — SARIF 2.1.0 export, idempotent (full overwrite), always written

Not for

  • Pure C / C++ codebases — use c-review instead.
  • Smart contracts (Solana, NEAR, Ink!) — use solana-vulnerability-scanner or the contract-specific skill.
  • Kernel-mode Rust without userspace allocator — coverage is incomplete; flag as advisory only.

References

Authors