mirror of
https://github.com/trailofbits/skills.git
synced 2026-09-14 14:28:48 +08:00
static-analysis: detect every language run-scans.sh can scan (#281)
* static-analysis: detect every language run-scans.sh can scan
Step 1 of the Semgrep scan globbed for 14 extensions while
`includes_for` in scripts/run-scans.sh carries globs for 41. Detection
is the only input to ruleset selection, so a category Step 1 never
reports is one Step 2 never selects and the scan never runs: YAML and
its Kubernetes, GitHub Actions, and CloudFormation variants, plus C#,
Kotlin, Scala, Swift, Elixir, Apex, and Solidity had rulesets in
references/rulesets.md that could not be reached. A header-only C++
tree (.h/.hh/.hpp/.hxx), a .cc-only one, and an ESM-only .mjs package
matched nothing either. The report read clean rather than incomplete.
The detection list is now the union of the `includes_for` globs, with a
note on the invariant and on the five types deliberately absent from
both because semgrep does not parse them. The category table gained the
rows the new patterns resolve to, so no glob dangles without a
destination; every category name is one `canonical_lang` folds. YAML
feeds four categories, so it carries disambiguation rules rather than a
single guess.
The framework markers were written without the `**/` prefix every
language pattern had, so they matched only the target root and a
monorepo keeping package.json in packages/*/ got no framework rulesets
at all. They are prefixed now, and pick up composer.json and
requirements.txt, which rulesets.md keys on but the list omitted.
workflows/semgrep-scan.js carried the same gap in its detect-phase
prompt. SKILL.md advertises it as the same scan without the approval
gate, so it gets the same category list, extension guidance, and YAML
disambiguation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* static-analysis: drop generic JSON from language detection
Globbing **/*.json detected a "json" category on essentially every
target: package.json, tsconfig.json, lockfiles and editor settings all
match. The only ruleset keyed to that category is r/json.aws, which
covers AWS IAM policy misconfigurations, so the effect was to attach an
IAM ruleset to every scan and to report a JSON language for projects
with no JSON worth scanning.
The detection list is therefore the union of the includes_for globs
minus that one glob, and the note says so rather than leaving the
"union" claim false. IAM policies and JSON-format CloudFormation
templates are still reachable by naming the json or cloudformation
category explicitly, which is the case where selecting r/json.aws is
what the user actually wants.
The dynamic workflow's detect prompt drops json from its category list
for the same reason, and says not to glob *.json, since a bare list
would invite the model to add it back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* static-analysis: assign JSON by content instead of dropping it
Review P2 on #281: removing **/*.json in 822f5da left r/json.aws and
JSON-format CloudFormation unreachable, and the remedy the note offered
— "name the json or cloudformation category explicitly" — has no input
path. parseArgs accepts only target, out, mode, jobs and skill; Select
sees only what Detect reported; and the gated plan lists only detected
categories, so nothing prompts the user either. A repo whose only IaC is
infra/template.json got neither ruleset, and because the category was
never planned it cannot appear in coveredNothing, failed or skipped. The
report reads clean, which is the failure mode this PR exists to remove.
So .json is globbed again, but the category is assigned from content
rather than from the extension, exactly as .yaml/.yml already is. That
keeps the reason it was dropped: package.json, tsconfig.json, lockfiles
and editor settings match no marker and yield no category, so an IAM
ruleset is not attached to every scan. AWSTemplateFormatVersion, or
Resources with a "Type": "AWS::" member, is cloudformation; a Statement
array whose elements have Effect is json. Sampling is steered at paths
that suggest infrastructure, since build config outnumbers policies.
Both extensions are now described as content-assigned in one place
rather than JSON being an exception to a union claim, so the detection
list is the full union of includes_for again — verified 41 = 41 with no
drift in either direction.
The dynamic workflow's detect prompt gets the same treatment; the review
noted it had the identical gap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* static-analysis: tell the detect phase to report cpp, not c
Review P3 on #281: the detect prompt names both `c` and `cpp` as
categories but gives their extensions as a single group, with nothing
saying which takes which. `includes_for c` is only `*.c *.h` while
`includes_for cpp` covers `.c .cc .cpp .cxx .h .hh .hpp .hxx`, so
reporting `c` for a .cpp/.hpp tree scans with --include=*.c
--include=*.h, opens zero files and exits 0. That lands in
coveredNothing rather than failing, so the C/C++ rules never read the
source and nothing says the scan was empty for the wrong reason.
cpp is a strict superset of c, so the rule is to report the superset.
The same relation holds for javascript and typescript — javascript
already carries the .ts/.tsx globs, which run-scans.sh notes at the
includes_for comment — so both pairs are stated together rather than as
one special case.
The gated path needs no change: scan-workflow.md's table folds all eight
extensions into one `C/C++` row, and canonical_lang maps `c/c++` to cpp,
so it already resolves to the superset. Verified rather than assumed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kz-tob
@@ -177,7 +177,7 @@
|
||||
},
|
||||
{
|
||||
"name": "static-analysis",
|
||||
"version": "1.4.1",
|
||||
"version": "1.4.2",
|
||||
"description": "Static analysis toolkit with CodeQL, Semgrep, and SARIF parsing for security vulnerability detection",
|
||||
"author": {
|
||||
"name": "Axel Mierczuk & Paweł Płatek"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "static-analysis",
|
||||
"version": "1.4.1",
|
||||
"version": "1.4.2",
|
||||
"description": "Static analysis toolkit with CodeQL, Semgrep, and SARIF parsing for security vulnerability detection",
|
||||
"author": {
|
||||
"name": "Axel Mierczuk & Paweł Płatek"
|
||||
|
||||
@@ -78,25 +78,54 @@ semgrep --pro --validate --metrics=off --config p/default 2>/dev/null && echo "P
|
||||
|
||||
**Detect languages** using Glob (not Bash). Run these patterns against the target directory and count matches:
|
||||
|
||||
`**/*.py`, `**/*.js`, `**/*.ts`, `**/*.tsx`, `**/*.jsx`, `**/*.go`, `**/*.rb`, `**/*.java`, `**/*.php`, `**/*.c`, `**/*.cpp`, `**/*.rs`, `**/Dockerfile`, `**/*.tf`
|
||||
`**/*.py`, `**/*.pyi`, `**/*.js`, `**/*.jsx`, `**/*.mjs`, `**/*.cjs`, `**/*.ts`, `**/*.tsx`, `**/*.go`, `**/*.rb`, `**/*.java`, `**/*.jsp`, `**/*.kt`, `**/*.kts`, `**/*.php`, `**/*.phtml`, `**/*.c`, `**/*.cc`, `**/*.cpp`, `**/*.cxx`, `**/*.h`, `**/*.hh`, `**/*.hpp`, `**/*.hxx`, `**/*.cs`, `**/*.rs`, `**/*.scala`, `**/*.swift`, `**/*.ex`, `**/*.exs`, `**/*.cls`, `**/*.trigger`, `**/*.sol`, `**/Dockerfile`, `**/*.dockerfile`, `**/*.tf`, `**/*.tfvars`, `**/*.hcl`, `**/*.yaml`, `**/*.yml`, `**/*.json`
|
||||
|
||||
Also check for framework markers: `package.json`, `pyproject.toml`, `Gemfile`, `go.mod`, `Cargo.toml`, `pom.xml`. Use Read to inspect these files for framework dependencies (e.g., read `package.json` to detect React, Express, Next.js; read `pyproject.toml` for Django, Flask, FastAPI).
|
||||
Step 2 can only select a ruleset for a category this step detected, so an extension missing here removes its ruleset from the scan with no signal — the report then reads clean rather than incomplete. The list is the union of the `includes_for` globs in [run-scans.sh](../scripts/run-scans.sh); keep the two in sync when either changes. `.mts`, `.cts`, `.C`, `Containerfile`, and `Dockerfile.prod` are absent from both, because semgrep does not parse them.
|
||||
|
||||
**Two extensions are matched by glob but assigned by content, not by extension.** `.yaml`/`.yml` and `.json` each feed several categories, and both are common in repositories that have no infrastructure to scan at all — nearly every project carries `package.json`, `tsconfig.json` and a lockfile. Assigning a category from the extension alone would attach an AWS IAM ruleset to every scan and report a JSON "language" for a project that has none. Assigning nothing would leave `r/json.aws` and JSON-format CloudFormation unreachable, which is worse: an unselected category never enters `rulesets.json`, so it cannot appear in `coveredNothing`, `failed` or `skipped` either, and the report reads clean. Glob for both, then read a sample and assign on the markers below.
|
||||
|
||||
Also check for framework markers: `**/package.json`, `**/pyproject.toml`, `**/requirements.txt`, `**/Gemfile`, `**/composer.json`, `**/go.mod`, `**/Cargo.toml`, `**/pom.xml`. Use Read to inspect these files for framework dependencies (e.g., read `package.json` to detect React, Express, Next.js; read `pyproject.toml` for Django, Flask, FastAPI). The `**/` prefix is required, not cosmetic: a bare `package.json` matches only the target root, so a monorepo with `packages/*/package.json` or `services/*/go.mod` gets no framework rulesets at all.
|
||||
|
||||
Map findings to categories:
|
||||
|
||||
| Detection | Category |
|
||||
|-----------|----------|
|
||||
| `.py`, `pyproject.toml` | Python |
|
||||
| `.js`, `.ts`, `package.json` | JavaScript/TypeScript |
|
||||
| `.py`, `.pyi`, `pyproject.toml`, `requirements.txt` | Python |
|
||||
| `.js`, `.jsx`, `.mjs`, `.cjs`, `.ts`, `.tsx`, `package.json` | JavaScript/TypeScript |
|
||||
| `.go`, `go.mod` | Go |
|
||||
| `.rb`, `Gemfile` | Ruby |
|
||||
| `.java`, `pom.xml` | Java |
|
||||
| `.php` | PHP |
|
||||
| `.c`, `.cpp` | C/C++ |
|
||||
| `.java`, `.jsp`, `pom.xml` | Java |
|
||||
| `.kt`, `.kts` | Kotlin |
|
||||
| `.php`, `.phtml`, `composer.json` | PHP |
|
||||
| `.c`, `.cc`, `.cpp`, `.cxx`, `.h`, `.hh`, `.hpp`, `.hxx` | C/C++ |
|
||||
| `.cs` | C# |
|
||||
| `.rs`, `Cargo.toml` | Rust |
|
||||
| `Dockerfile` | Docker |
|
||||
| `.tf` | Terraform |
|
||||
| k8s manifests | Kubernetes |
|
||||
| `.scala` | Scala |
|
||||
| `.swift` | Swift |
|
||||
| `.ex`, `.exs` | Elixir |
|
||||
| `.cls`, `.trigger` | Apex |
|
||||
| `.sol` | Solidity |
|
||||
| `Dockerfile`, `.dockerfile` | Docker |
|
||||
| `.tf`, `.tfvars`, `.hcl` | Terraform |
|
||||
| `.yaml`, `.yml` | YAML, Kubernetes, GitHub Actions, or CloudFormation — disambiguate below |
|
||||
| `.json` | CloudFormation or JSON, or no category at all — disambiguate below |
|
||||
|
||||
**Disambiguating YAML.** One `.yaml`/`.yml` match feeds four categories, so Read a sample of the matches before assigning:
|
||||
|
||||
- path under `.github/workflows/` → GitHub Actions
|
||||
- `apiVersion:` together with `kind:` → Kubernetes
|
||||
- `AWSTemplateFormatVersion:`, or `Resources:` with a `Type: AWS::` member → CloudFormation
|
||||
- anything else → YAML
|
||||
|
||||
These are not exclusive; assign every category that matches. Include the generic YAML category whenever any YAML is present, since `p/yaml` carries patterns the specific rulesets do not.
|
||||
|
||||
**Disambiguating JSON.** Unlike YAML, `.json` has no catch-all: most JSON in a repository is build configuration that no ruleset covers, so the default is to assign nothing. Read a sample and assign only on these markers:
|
||||
|
||||
- `"AWSTemplateFormatVersion"`, or `"Resources"` whose members carry a `"Type": "AWS::…"` → CloudFormation
|
||||
- a `"Statement"` array whose elements have `"Effect"` → JSON (this is the IAM policy shape `r/json.aws` targets)
|
||||
- anything else, including `package.json`, `tsconfig.json`, `composer.json`, lockfiles and editor settings → **no category**
|
||||
|
||||
Do not report a `json` category because JSON files exist. Report it when a sampled file has the IAM policy shape. Prefer sampling files whose path suggests infrastructure — `iam/`, `policies/`, `cloudformation/`, `infra/`, `*template*.json` — since a repository with thousands of JSON files will have its IAM policies outnumbered by build configuration, and a sample drawn without regard to path is likely to miss them.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -239,13 +239,45 @@ const detected = await agent(
|
||||
' p/default against the registry, so without it an audit phones home before scanning.',
|
||||
'',
|
||||
'6. Detect languages by counting files. Report the count that justified each category, since',
|
||||
' a category with one file is worth knowing about before its rulesets run. Cover at least:',
|
||||
' python, javascript, typescript, go, ruby, java, php, c, cpp, rust, docker, terraform,',
|
||||
' kubernetes. Use lowercase category names.',
|
||||
' a category with one file is worth knowing about before its rulesets run. Cover every',
|
||||
' category run-scans.sh has globs for, because one it never hears about is one it never',
|
||||
' scans, and the report then reads clean rather than incomplete:',
|
||||
' python, javascript, typescript, go, ruby, java, kotlin, php, c, cpp, csharp, rust,',
|
||||
' scala, swift, elixir, apex, solidity, docker, terraform, json, cloudformation,',
|
||||
' github-actions, kubernetes, yaml. Use lowercase category names.',
|
||||
' Search by extension, not by name alone: .py .pyi / .js .jsx .mjs .cjs / .ts .tsx / .go /',
|
||||
' .rb / .java .jsp / .kt .kts / .php .phtml / .c .cc .cpp .cxx .h .hh .hpp .hxx / .cs /',
|
||||
' .rs / .scala / .swift / .ex .exs / .cls .trigger / .sol / Dockerfile *.dockerfile /',
|
||||
' .tf .tfvars .hcl / .yaml .yml / .json. A header-only C++ tree or an ESM-only .mjs',
|
||||
' package matches none of the obvious extensions.',
|
||||
' Two pairs of categories OVERLAP, and the extension group above does not say which takes',
|
||||
' which. Report the SUPERSET, never the narrower one:',
|
||||
' c vs cpp — report cpp unless the tree is .c/.h only. `includes_for cpp` covers',
|
||||
' .c .cc .cpp .cxx .h .hh .hpp .hxx, while `includes_for c` is only',
|
||||
' .c .h, so choosing c for a .cpp/.hpp tree scans with --include=*.c',
|
||||
' --include=*.h, opens zero files and exits 0. The run lands in',
|
||||
' coveredNothing rather than failing, so the C/C++ rules never read the',
|
||||
' source and nothing says the scan was empty for the wrong reason.',
|
||||
' javascript vs typescript — javascript already carries the .ts/.tsx globs, so report',
|
||||
' javascript for a mixed or TS-only tree. Reporting both scans the same',
|
||||
' files twice under two rulesets.',
|
||||
' YAML and JSON are assigned by CONTENT, not by extension: both are everywhere, and both',
|
||||
' carry rulesets that must stay reachable. Glob for them, then Grep or read a sample.',
|
||||
' YAML feeds four categories — under .github/workflows/ is github-actions, apiVersion+kind',
|
||||
' is kubernetes, AWSTemplateFormatVersion or Resources with Type: AWS:: is cloudformation,',
|
||||
' anything else is yaml. They are not exclusive; report each that matches.',
|
||||
' JSON has no catch-all: "AWSTemplateFormatVersion", or "Resources" with a "Type": "AWS::"',
|
||||
' member, is cloudformation; a "Statement" array whose elements have "Effect" is json (the',
|
||||
' IAM policy shape r/json.aws targets); anything else — package.json, tsconfig.json,',
|
||||
' lockfiles, editor settings — is NO category. Do not report json merely because .json',
|
||||
' files exist, and prefer sampling paths that suggest infrastructure (iam/, policies/,',
|
||||
' cloudformation/, infra/, *template*.json), since build config will outnumber policies.',
|
||||
'',
|
||||
'7. Read the framework markers that exist — package.json, pyproject.toml, Gemfile, go.mod,',
|
||||
' Cargo.toml, pom.xml — and name the frameworks you find (django, flask, react, express,',
|
||||
' nextjs, spring, …). These select extra rulesets in the next phase.',
|
||||
'7. Read the framework markers that exist — package.json, pyproject.toml, requirements.txt,',
|
||||
' Gemfile, composer.json, go.mod, Cargo.toml, pom.xml — and name the frameworks you find',
|
||||
' (django, flask, react, express, nextjs, spring, laravel, …). Look at every level, not',
|
||||
' just the target root: a monorepo keeps them in packages/*/ or services/*/. These select',
|
||||
' extra rulesets in the next phase.',
|
||||
'',
|
||||
'Report a language only when files actually matched. An invented category costs a ruleset',
|
||||
'that scans nothing and reads in the final report as coverage that happened.',
|
||||
|
||||
Reference in New Issue
Block a user