Files
trailofbits__skills/plugins/constant-time-analysis
Dan Guido 9e06dc67a3 Make every documented command runnable under our own python shims (#258)
* Make every documented command runnable under our own python shims

The modern-python plugin ships PATH shims that refuse `python <script>`,
`pip install`, `python -m pip` and `uv pip install`. Twelve other plugins
in this marketplace issued exactly those forms, so installing our own
plugin broke our own skills — and CI was green throughout.

The worst case was not theoretical. c-review and rust-review both call
their Phase 4 planner as `python3 "${PLUGIN_ROOT}/scripts/build_run_plan.py"`,
so with the shim installed every run died before spawning a worker.
Verified both directions: the new form exits 0 with the shim on PATH, the
old form exits 1.

Phase 1's reading pass named 16 skills. A mechanical sweep found 96
candidate lines across 44 files, and scanning shell scripts as well as
markdown found 10 more the docs sweep had missed. That gap is the reason
the check below exists.

The fix is not one substitution. Four classes needed different treatment:

- Our own scripts become `uv run --no-project <script>`. Not bare `uv run`,
  because these execute inside the *target* repo, which may be a Python
  project that cannot resolve; verified against a broken pyproject.toml and
  against validate_artifacts.py's sibling import of generate_sarif.
- Package installs become `uv add` for a dependency, `uv tool install` for a
  CLI, `uv sync` for a project's own editable install.
- Third-party CLIs we merely document — OSS-Fuzz's infra/helper.py, yarGen —
  become `uv run --no-project python <script>`, which keeps upstream's exact
  semantics rather than handing their script an environment we manage.
- atheris's instrumented build keeps its source build, as
  `uv add --no-binary-package cbor2`. Dropping that flag would silently
  produce an uninstrumented fuzzer, which is worse than a visible failure.
  Its prose was updated to name the flag it now uses.

Two factual corrections fell out. `pip install caracal` was wrong twice
over: caracal is a Rust tool (Cargo.toml at its root), so it is now
upstream's own `cargo install --git`, not a uv equivalent that would fetch
an unrelated PyPI package. And `pip install uv` cannot bootstrap uv under
a shim that intercepts pip, so culture-index now points at the official
installer.

Thirteen lines stay as they are, each deliberately: Dockerfile `RUN` lines
and oss-fuzz's build.sh run in containers where our shims are absent;
codeql's pip calls install the *analysed* project's dependencies, and that
project is arbitrary; trailmark's dispatch skills must keep saying "Do NOT
run `pip install`"; and modern-python documents what it intercepts.

`make shell-suites` passes again as a result — exit 0 with the 1.6.0 shim,
where AGENTS.md previously recorded it as broken by variant-analysis.

The guardrail: check_python_invocations scans 698 markdown and shell files
and fails on the four refused forms, with structural exemptions for
dockerfile fences and an `allow-legacy-python: <reason>` marker that scopes
to its code block. Eleven self-test fixtures cover it, four asserting it
fires and seven asserting it stays quiet on the compliant forms. It was
mutation-tested in both languages, and it caught its own worst bug during
development: unanchored patterns first flagged `uv run --no-project python
fuzz.py`, the very form the advice recommends. Self-test goes 45 -> 56.

* Review pass: fix the atheris flow, drop a stray exemption, trim comments

Three corrections from reviewing the branch diff:

- atheris's install now opens with `uv init --bare`, without which the
  documented `uv add atheris` errors in a bare harness directory. The old
  pip form assumed an activated venv, so setup was always implicit; now
  it is one explicit line.
- ossfuzz carried an allow-legacy-python marker on a C++ build block that
  contains no python at all — yesterday's insertion matched the first of
  three "Build in build.sh" headings instead of the python one. The
  exemption now sits only on the block that needs it.
- The anti-vacuity message said "read no markdown" for a scan that also
  covers shell scripts.

The rest is weight: the new check's comment blocks, the hardcoded-path
constants' commentary, the AGENTS.md bullets and the three exemption
markers all said the same things at two to three times the length. Each
keeps its one-line why; the narratives are gone. No behavioural change —
self-test still passes 56 assertions and the full scan is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Address the review: fix where packages land, widen the check to match the shim

The review's core insight was right twice over. Several substitutions had
changed WHERE a package lands, breaking the documented next step, and the
checker enforced a narrower invariant than the shim it exists to mirror.

Where packages land:

- trailmark is imported as a library from five skills, and a `uv tool
  install` environment is not importable — the retry loop at
  trailmark/SKILL.md:47-51 would have spun forever on the exact error it
  names. The CLI install stays `uv tool install`; the import snippets now
  run under `uv run --with trailmark python -`.
- `uv add` writes to the manifest of whatever project you are standing
  in, which for sarif-parsing is the audited repo. Its scripting rows,
  ijson comment and jsonschema example now use `uv run --with <pkg>`,
  which leaves no trace. atheris keeps `uv add` deliberately: the fuzzing
  harness is the user's own project, made explicit by `uv init --bare`.
- `uv sync` leaves ct-analyzer in .venv/bin, so the README's very next
  line failed with command not found. Now `uv tool install .`, verified
  end to end: the console script lands on PATH and --help runs.
- yarGen needs pefile/lxml/yara-python, which `--no-project` had detached;
  now `uv run --with-requirements requirements.txt`.
- The cbor2 source-build preference now persists via
  `no-binary-package = ["cbor2"]` under [tool.uv] (field verified against
  uv's accepted-settings list), so a later `uv sync` cannot silently swap
  in an uninstrumented wheel.

The checker, widened to the shim's actual behaviour:

- `python3 --version` and `python3 -u foo.py` are refused by the shim but
  passed the old patterns; one live instance (constant-time-analysis
  README) proved it. Both forms are now caught.
- Every `uv pip` subcommand is refused, not just install; `-t` joins the
  allowed tool-managed flags.
- .py files are scanned too: usage strings and error messages told users
  to run refused commands from ten scripts, including the --help of the
  very planner this PR fixed. All rewritten.
- The evals/tests exemption now tests path parts relative to plugins/, so
  a checkout under a directory named tests no longer exempts every file.
- An allow-marker's scope ends at a blank line as well as a fence, so one
  marker cannot blanket a whole file; quality-assessment.md gains the
  second marker that scoping made necessary.

Also from the review: zeroize's preflight gets `which python3` back (a
helper script still needs the binary; the shim never required removing
it), the Makefile's shell-suites note no longer describes an interception
that is gone, and the cairo CI example warns that it rebuilds caracal
from source each run.

Self-test 56 -> 63; every new pattern and exemption is fixture-covered
and was mutation-probed against the real tree. Full scan: 0 findings over
773 files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Address the second review: prerequisite probe, checker parity, package placement

The review's P2 was a regression this PR introduced for a population the
first fix ignored: c-review and rust-review now require uv, and a box
with python3 but no uv would die at Phase 4 exactly the way shimmed boxes
died before. Phase 1 (Prerequisites) in both skills now probes
`command -v uv` and aborts with install guidance. zeroize-audit's
preflight already checked uv. The four converted shell suites gain the
same guard with a clear message instead of a bare 127 mid-run.

Checker parity with the shims, second pass:

- pipx and the non-install pip subcommands are refused by catch-all shim
  arms and passed the checker; both get named-subcommand patterns.
- A script named by variable or path (`python3 "$MERGE"`) has no `.py`
  token; a new pattern covers it and immediately caught one live
  instance — a codeql test stub that fakes uv itself, now carrying an
  allow-marker with its reason.
- finditer everywhere: a compliant `uv run` earlier on a line no longer
  masks a refused command later on it, which was exactly the table-cell
  case the unanchored design exists for.
- Prohibition phrases now test the text BEFORE the match, so
  "Use `pip install semgrep` instead of the tarball" is flagged while
  "Do NOT run `pip install`" stays exempt.
- The uv-pip allowance matches whole flags after the command, so
  `--target-dir` no longer counts as `--target` and a trailing `-t /tmp`
  does; `uv pip` precedes `pip` in the pattern order so its lines get
  the right advice; a pip match directly after `uv ` defers to the
  uv-pip verdict instead of double-reporting.

Package placement, continued from the same insight as round one:

- yarGen regains --no-project alongside --with-requirements, plus a cd
  into the checkout so requirements.txt resolves where it lives.
- sarif-parsing's jsonschema example no longer names a script that does
  not exist, and the table's run-forms show a concrete script.py.
- culture-index's two messages now agree and name the actual remedy
  (`uv run --project` on the scripts directory) instead of re-adding a
  dependency its pyproject already declares.
- merge_sarif's usage line gains --no-project; the generator plugin's
  install section stops prescribing a venv its own runner never uses.
- generate_poc declared requires-python >=3.9 while using `str | None`
  in a signature, a TypeError on 3.9 that uv's interpreter selection
  made reachable; now >=3.10.
- The GitLab CI example exports ~/.local/bin onto PATH, without which
  `uv tool install` warns and the next line dies command-not-found.

Self-test 63 -> 71; the masking, prohibition-direction, flag-position
and pipx cases are all fixtures, and each new pattern was probed live
against the tree (plant, error, remove, clean — 0 findings over 773
files).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Address the third review: importable trailmark, honest probes, sturdier scan

The P2 was the residue of round two's own fix, applied to the siblings
but not the flagship: trailmark/SKILL.md told the model to cure an import
error with `uv tool install`, which cannot cure it — a tool env is not
importable — while forbidding every fallback. The install block now says
what each remedy is for: `uv tool install` for the CLI, `uv run --with
trailmark python -` for the snippets, and the other five library-first
docs carry the same one-line annotation next to their install command.

Empirically settled rather than taken from the review: `uv run python3
<script>` works fine under the shims — uv prepends its environment's bin
directory, so python3 resolves to a real interpreter, not the shim. The
review's claim to the contrary would have meant rewriting the Makefile
and a bats suite; a two-minute transcript said no. Also declined: a
zeroize uv-prerequisite (its preflight already lists uv and uvx; the
C/C++ `which` line now names uv too).

Real and fixed:

- ct-analyzer's availability probe ran `python3 --version` by subprocess
  — the one refused form — so under the shims it reported "Python is not
  available" on machines where it plainly is. It now probes
  sys.executable, the interpreter the analyzer itself runs under.
  Verified under the shim: probe returns True.
- The flag step-over in both script patterns handles long and
  value-taking flags (`python3 -W ignore harness.py`, `--verbose
  tool.py`), matching the shim's two-slot consumption.
- A bare `allow-legacy-python:` with no reason no longer exempts
  anything; the reason the docs demand is now enforced.
- `uv run {baseDir}/...` gets --no-project at the ten semgrep and
  culture-index call sites that round two missed, and the culture-index
  remediation strings now name that same runnable command instead of a
  --project mechanism nothing uses.
- pip gains cache/config; the pattern comment now says the subcommand
  list is deliberately a subset.
- Both filesystem scans skip .venv/node_modules-style directories, after
  a stray local .venv (left by this session's own uv probe, and invisible
  to CI) turned the path scan red.

Smaller review items: the uv-probe prose says "Phase 4 onward" rather
than a wrong phase range, run_fixtures' comment stops claiming PEP 723
headers its stdlib-only helpers do not have, the yarGen one-liners say
to run from the checkout, `uv tool install` sites note or export the
tool bin dir the way a fresh container needs, and sarif-parsing's table
column says Install / run and stops naming a file that does not exist.

Self-test 71 -> 74. Full scan: 0 findings over 773 files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 13:39:07 -04:00
..

Constant-Time Analyzer (ct-analyzer)

A portable tool for detecting timing side-channel vulnerabilities in compiled cryptographic code. Analyzes assembly output from multiple compilers and architectures to detect instructions that could leak secret data through execution timing.

Background

Timing side-channel attacks exploit variations in execution time to extract secret information from cryptographic implementations. Common sources include:

  • Hardware division (DIV, IDIV): Execution time varies based on operand values
  • Floating-point operations (FDIV, FSQRT): Variable latency based on inputs
  • Conditional branches: Different execution paths have different timing

The infamous KyberSlash attack demonstrated how division instructions in post-quantum cryptographic implementations could be exploited to recover secret keys.

Features

  • Multi-language support: C, C++, Go, Rust, Swift, Java, Kotlin, C#, PHP, JavaScript, TypeScript, Python, Ruby
  • Multi-architecture support: x86_64, ARM64, ARM, RISC-V, PowerPC, s390x, i386
  • Multi-compiler support: GCC, Clang, Go compiler, Rustc, Swiftc
  • Bytecode support: Java/Kotlin (JVM), C# (CIL), PHP (VLD/opcache), JavaScript/TypeScript (V8 bytecode), Python (dis), Ruby (YARV)
  • Optimization-level testing: Test across O0-O3, Os, Oz
  • Multiple output formats: Text, JSON, GitHub Actions annotations
  • Cross-compilation: Analyze code for different target architectures

Quick Start

# Install
uv tool install .

# Analyze a C file
ct-analyzer crypto.c

Usage

Basic Analysis

ct-analyzer <source_file>

Options

Option Description
--arch, -a Target architecture (x86_64, arm64, arm, riscv64, ppc64le, s390x, i386)
--compiler, -c Compiler to use (gcc, clang, go, rustc)
--opt-level, -O Optimization level (O0, O1, O2, O3, Os, Oz) - default: O2
--warnings, -w Include conditional branch warnings
--func, -f Regex pattern to filter functions
--json Output JSON format
--github Output GitHub Actions annotations
--list-arch List supported architectures

Examples

# Test with different optimization levels
ct-analyzer --opt-level O0 crypto.c
ct-analyzer --opt-level O3 crypto.c

# Cross-compile for ARM64
ct-analyzer --arch arm64 crypto.c

# Include conditional branch warnings
ct-analyzer --warnings crypto.c

# Analyze specific functions
ct-analyzer --func 'decompose|sign' crypto.c

# JSON output for CI
ct-analyzer --json crypto.c

# Analyze Go code
ct-analyzer crypto.go

# Analyze Rust code
ct-analyzer crypto.rs

# Analyze PHP code (requires PHP with VLD extension or opcache)
ct-analyzer crypto.php

# Analyze TypeScript (transpiles to JS first)
ct-analyzer crypto.ts

# Analyze JavaScript (uses V8 bytecode analysis)
ct-analyzer crypto.js

# Analyze Python (uses dis module for bytecode disassembly)
ct-analyzer crypto.py

# Analyze Ruby (uses YARV instruction dump)
ct-analyzer crypto.rb

Detected Vulnerabilities

Error-Level (Must Fix)

Category x86_64 ARM64 RISC-V
Integer Division DIV, IDIV, DIVQ, IDIVQ UDIV, SDIV DIV, DIVU, REM, REMU
FP Division DIVSS, DIVSD, DIVPS, DIVPD FDIV FDIV.S, FDIV.D
Square Root SQRTSS, SQRTSD, SQRTPS, SQRTPD FSQRT FSQRT.S, FSQRT.D

Warning-Level (Review Needed)

Conditional branches that may leak timing if condition depends on secret data:

  • x86: JE, JNE, JZ, JNZ, JA, JB, JG, JL, etc.
  • ARM: BEQ, BNE, CBZ, CBNZ, TBZ, TBNZ
  • RISC-V: BEQ, BNE, BLT, BGE

Scripting Language Support

PHP Analysis

PHP analysis uses either the VLD extension (recommended) or opcache debug output:

Detected PHP Vulnerabilities:

Category Pattern Recommendation
Division ZEND_DIV, ZEND_MOD Use Barrett reduction
Cache timing chr(), ord() Use pack('C', $int) / unpack('C', $char)[1]
Table lookups bin2hex(), hex2bin(), base64_encode() Use constant-time alternatives
Array access FETCH_DIM_R (secret index) Use constant-time table lookup
Bit shifts ZEND_SL, ZEND_SR (secret amount) Mask shift amount
Variable encoding pack(), serialize(), json_encode() Use fixed-length output
Weak RNG rand(), mt_rand(), uniqid() Use random_int() / random_bytes()
String comparison strcmp(), === on secrets Use hash_equals()

Installation:

# Install VLD extension (recommended)
# Query latest version from PECL
VLD_VERSION=$(curl -s https://pecl.php.net/package/vld | grep -oP 'vld-\K[0-9.]+(?=\.tgz)' | head -1)
pecl install channel://pecl.php.net/vld-${VLD_VERSION}

# Or build from source (if PECL fails)
git clone https://github.com/derickr/vld.git && cd vld
phpize && ./configure && make && sudo make install

# Or use opcache (built-in, fallback)
# Enabled by default in PHP 7+

JavaScript/TypeScript Analysis

JavaScript analysis uses V8 bytecode via Node.js --print-bytecode. TypeScript files are automatically transpiled first.

Detected JS Vulnerabilities:

Category Pattern Recommendation
Division Div, Mod bytecodes Use constant-time multiply-shift
Array access LdaKeyedProperty (secret index) Use constant-time table lookup
Bit shifts ShiftLeft, ShiftRight (secret amount) Mask shift amount
Variable encoding TextEncoder, JSON.stringify(), btoa() Use fixed-length output
Weak RNG Math.random() Use crypto.getRandomValues() or crypto.randomBytes()
Variable latency Math.sqrt(), Math.pow() Avoid in crypto paths
String comparison === on secrets Use crypto.timingSafeEqual() (Node.js)
Early-exit search indexOf(), includes() Use constant-time comparison

Requirements:

# Node.js required
node --version

# TypeScript compiler (optional, for .ts files)
npm install -g typescript

Python Analysis

Python analysis uses the built-in dis module to analyze CPython bytecode.

Detected Python Vulnerabilities:

Category Pattern Recommendation
Division BINARY_OP 11 (/), BINARY_OP 6 (%) Use Barrett reduction or constant-time alternatives
Array access BINARY_SUBSCR (secret index) Use constant-time table lookup
Bit shifts BINARY_LSHIFT, BINARY_RSHIFT (secret amount) Mask shift amount
Variable encoding int.to_bytes(), json.dumps(), base64.b64encode() Use fixed-length output
Weak RNG random.random(), random.randint() Use secrets.token_bytes() / secrets.randbelow()
Variable latency math.sqrt(), math.pow() Avoid in crypto paths
String comparison == on secrets Use hmac.compare_digest()
Early-exit search .find(), .startswith() Use constant-time comparison

Requirements:

# Python 3.x required (built-in dis module)
uv run python --version

Ruby Analysis

Ruby analysis uses YARV (Yet Another Ruby VM) bytecode via ruby --dump=insns.

Detected Ruby Vulnerabilities:

Category Pattern Recommendation
Division opt_div, opt_mod Use constant-time alternatives
Array access opt_aref (secret index) Use constant-time table lookup
Bit shifts opt_lshift, opt_rshift (secret amount) Mask shift amount
Variable encoding pack(), to_json(), Base64.encode64() Use fixed-length output
Weak RNG rand(), Random.new Use SecureRandom.random_bytes()
Variable latency Math.sqrt() Avoid in crypto paths
String comparison == on secrets Use Rack::Utils.secure_compare() or OpenSSL
Early-exit search .include?(), .start_with?() Use constant-time comparison

Requirements:

# Ruby required (YARV is standard since Ruby 1.9)
ruby --version

Example Output

============================================================
Constant-Time Analysis Report
============================================================
Source: decompose.c
Architecture: arm64
Compiler: clang
Optimization: O2
Functions analyzed: 4
Instructions analyzed: 88

VIOLATIONS FOUND:
----------------------------------------
[ERROR] SDIV
  Function: decompose_vulnerable
  Reason: SDIV has early termination optimization; execution time depends on operand values

[ERROR] SDIV
  Function: use_hint_vulnerable
  Reason: SDIV has early termination optimization; execution time depends on operand values

----------------------------------------
Result: FAILED
Errors: 2, Warnings: 0

Fixing Violations

Replace Division with Barrett Reduction

// VULNERABLE
int32_t q = a / divisor;

// SAFE: Barrett reduction
// Precompute: mu = ceil(2^32 / divisor)
uint32_t q = (uint32_t)(((uint64_t)a * mu) >> 32);

Replace Branches with Constant-Time Selection

// VULNERABLE
if (secret) {
    result = a;
} else {
    result = b;
}

// SAFE: Constant-time selection
uint32_t mask = -(uint32_t)(secret != 0);
result = (a & mask) | (b & ~mask);

Replace Comparisons

// VULNERABLE
if (memcmp(a, b, len) == 0) { ... }

// SAFE: Use crypto/subtle or equivalent
if (subtle.ConstantTimeCompare(a, b) == 1) { ... }

Test Samples

The repository includes test samples demonstrating vulnerable and secure implementations:

  • ct_analyzer/tests/test_samples/decompose_vulnerable.c - Vulnerable C implementation
  • ct_analyzer/tests/test_samples/decompose_constant_time.c - Constant-time C implementation
  • ct_analyzer/tests/test_samples/decompose_vulnerable.go - Vulnerable Go implementation
  • ct_analyzer/tests/test_samples/decompose_vulnerable.rs - Vulnerable Rust implementation
  • ct_analyzer/tests/test_samples/vulnerable.php - Vulnerable PHP implementation
  • ct_analyzer/tests/test_samples/vulnerable.ts - Vulnerable TypeScript implementation
  • ct_analyzer/tests/test_samples/vulnerable.py - Vulnerable Python implementation
  • ct_analyzer/tests/test_samples/vulnerable.rb - Vulnerable Ruby implementation

These implement the Decompose and UseHint algorithms from ML-DSA (FIPS-204) as test cases.

CI Integration

GitHub Actions

name: Constant-Time Check

on: [push, pull_request]

jobs:
  ct-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: |
          uv tool install .
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"

      - name: Check constant-time properties
        run: |
          ct-analyzer --github src/crypto/*.c

GitLab CI

ct-check:
  stage: test
  script:
    - uv tool install .
    - export PATH="$HOME/.local/bin:$PATH"
    - ct-analyzer --json src/crypto/*.c > ct-report.json
  artifacts:
    reports:
      codequality: ct-report.json

Limitations

  1. Compiler Output Analysis: Analyzes what the compiler produces, not runtime behavior. Cannot detect:

    • Cache timing attacks from memory access patterns
    • Microarchitectural side-channels (Spectre, etc.)
    • Processor-specific optimizations
  2. No Data Flow Analysis: Flags all dangerous instructions regardless of whether they operate on secret data. Manual review is needed to determine if flagged code handles secrets. This means false positives are expected - for example, division used in loop bounds with public constants will be flagged even though it's not a vulnerability.

  3. False Positive Verification: For each flagged violation, verify the operands:

    • If operands are compile-time constants or public parameters → likely false positive
    • If operands are derived from keys, plaintext, or secrets → true positive
    • See the SKILL.md documentation for detailed triage guidance
  4. Compiler Variations: Different compilers/versions may produce different assembly. Test with:

    • Multiple optimization levels
    • Multiple compilers
    • Target production architectures
  5. Scripting Languages: PHP, JavaScript/TypeScript, Python, and Ruby are supported via bytecode analysis.

Running Tests

uv run python ct_analyzer/tests/test_analyzer.py

References

Acknowledgments

Based on the test_ct utility created for ML-DSA.