# jscpd v5 (Rust Engine) jscpd v5 is a Rust engine shipped as a self-contained binary: no runtime required, same CLI flags, reporters and `.jscpd.json` config as the earlier Node.js versions. It is distributed under three names: | Package | Installs commands | Notes | |---------|-------------------|-------| | [`jscpd`](https://www.npmjs.com/package/jscpd) | `jscpd` | Default install; installs the `jscpd` command | | [`cpd`](https://www.npmjs.com/package/cpd) | `cpd` | Lighter package, shorter command only | | [`jscpd` (crates.io)](https://crates.io/crates/jscpd) | `jscpd` **and** `cpd` | Rust-native install; both binaries | All three install the identical Rust binary and accept the same CLI options. Only the crates.io install exposes both command names from a single package. ## Performance A comparison against other copy/paste detectors (jscpd-rs, Duplo, Fallow, Simian, PMD CPD) on the repository's `fixtures/` corpus — timing, detection counts, cross-format detection and AI-token efficiency — is in [benchmark/BENCHMARK.md](../benchmark/BENCHMARK.md). ## Installation ```bash # npm — installs the jscpd command npm install -g jscpd jscpd /path/to/code # npm — installs only the cpd command (lighter) npm install -g cpd cpd /path/to/code # crates.io — Rust-native install (exposes both jscpd and cpd commands) cargo install jscpd jscpd /path/to/code cpd /path/to/code # Nix — run without installing nix run github:kucherenko/jscpd -- /path/to/code # Nix — install permanently nix profile install github:kucherenko/jscpd # Homebrew (macOS/Linux) brew install jscpd # PyPI — platform wheels with the same binary (pip, pipx, uv) pip install jscpd uvx jscpd /path/to/code ``` The npm packages ship prebuilt binaries for 8 platforms — no Node.js runtime is required, the binary is self-contained: | Platform | npm package | Rust target | |----------|-------------|-------------| | macOS arm64 | `jscpd-darwin-arm64` | `aarch64-apple-darwin` | | macOS x64 | `jscpd-darwin-x64` | `x86_64-apple-darwin` | | Linux arm64 (glibc) | `jscpd-linux-arm64-gnu` | `aarch64-unknown-linux-gnu` | | Linux arm64 (musl) | `jscpd-linux-arm64-musl` | `aarch64-unknown-linux-musl` | | Linux x64 (glibc) | `jscpd-linux-x64-gnu` | `x86_64-unknown-linux-gnu` | | Linux x64 (musl) | `jscpd-linux-x64-musl` | `x86_64-unknown-linux-musl` | | Windows arm64 | `jscpd-windows-arm64-msvc` | `aarch64-pc-windows-msvc` | | Windows x64 | `jscpd-windows-x64-msvc` | `x86_64-pc-windows-msvc` | The same binaries are attached to every [GitHub Release](https://github.com/kucherenko/jscpd/releases) as `jscpd-.tar.gz` with a `checksums.txt` and SLSA provenance, and packaged as a multi-arch Docker image at `ghcr.io/kucherenko/jscpd` (see [CI docs](ci-and-hooks.md#docker)). ## CLI Usage The `jscpd` command is available after installing `jscpd` from npm; the `cpd` command is available after installing either `cpd` (npm) or `jscpd` (crates.io). Both commands accept the same options and are identical: ```bash jscpd [OPTIONS] [PATH]... jscpd [OPTIONS] [PATH]... ``` ### Options | Option | Short | Description | Default | |--------|-------|-------------|---------| | `--min-tokens` | `-k` | Minimum tokens in a clone | 50 | | `--min-lines` | `-l` | Minimum lines in a clone | 5 | | `--max-lines` | `-x` | Maximum source file lines | — | | `--max-size` | `-z` | Skip files larger than SIZE (e.g. `1kb`, `1mb`, `100kb`) | no limit | | `--mode` | `-m` | Detection mode: `mild`, `weak`, `strict` | `mild` | | `--ignore-pattern` | | Comma-separated regular expressions; source text matched by a pattern is excluded from clone detection. See [Ignoring source regions](#ignoring-source-regions) | — | | `--workers` | | Number of worker threads for parallel tokenization/detection | auto (all CPU cores) | | `--no-colors` | | Disable ANSI color output | off | | `--absolute` | `-a` | Use absolute paths in reports | off | | `--follow-symlinks` | | Follow symbolic links while walking. A file reached through a link is reported by the path it was found at (relative to the scan root, like any other file), and a file reachable through several paths is scanned once. v4 followed links by default | off | | `--ignore-case` | | Ignore case of symbols in code (experimental) | off | | `--ignore-identifiers` | | Treat all identifiers as equal, so clones that differ only in variable, function or type names are found. See [Type-2 clones](#type-2-clones-renamed-identifiers-literals-and-annotations) | off | | `--ignore-literals` | | Treat all string literals as equal and all numeric literals as equal | off | | `--ignore-annotations` | | Skip annotations and decorators (`@Name`, `@Name(...)`) before detection | off | | `--max-gap-lines` | | Merge clones of one file pair separated by at most N unmatched lines in both files into one near-miss clone reported as `similar`. See [Type-3 clones](#type-3-clones-near-miss-merging-with---max-gap-lines) | 0 (off) | | `--similarity` | | Report JavaScript/TypeScript function pairs whose syntax-tree similarity reaches RATIO, a number in `(0, 1]`, as `similar` clones; `1` means exact matches only. See [function similarity](#function-level-similarity-with---similarity) | 1 (off) | | `--kind` | | Report only clones of these kinds, comma-separated: `exact`, `renamed`, `similar`, `gap`, `ast`. See [Filtering by kind](#filtering-by-kind-with---kind) | all | | `--formats-exts` | | Custom format-to-extension mapping (e.g. `javascript:es,es6;dart:dt`) | — | | `--formats-names` | | Custom format-to-filename mapping | — | | `--cross-formats` | | Detect clones across formats: `;`-separated groups of `,`-separated formats (e.g. `javascript,typescript`). Preset `js-ts` = `javascript,jsx,typescript,tsx` | — | | `--list` | | List all supported formats and exit | — | | `--skip-local` | | Skip clones where both fragments are in the same directory | off | | `--skip-isolated` | | Skip clones between different folders of the same isolation group: `,`-separated groups of `\|`-separated folders (e.g. `packages/a\|packages/b`). Useful in monorepos where teams own separate packages | — | | `--baseline` | | Clone baseline file (e.g. `.jscpd-baseline.json`): clones whose fingerprint is absent from it are reported as new. See [Baseline](#baseline) | — | | `--update-baseline` | | Rewrite the baseline file from the current run, creating it if missing (requires `--baseline`) | off | | `--fail-on-new-clones` | | Exit 1 when more than N new clones are found (`--fail-on-new-clones` alone means N=0; requires `--baseline` or `--baseline-from-ref`) | — | | `--fail-on-empty` | | Exit 1 when the scan analyzes no files: the paths exist but nothing matched the `--format`, `--ignore` and `--pattern` filters, or every file was below `--min-tokens`. See [Exit codes](#exit-codes) | off | | `--baseline-from-ref` | | Compare against an ephemeral baseline built from a git ref's tree (e.g. `origin/main`). Conflicts with `--baseline` | — | | `--sarif-error-tokens` | | Report SARIF results as `error` for clones with at least this many tokens (smaller clones stay `warning`). When overall duplication exceeds `--threshold`, all SARIF results become `error` regardless of size. | — (all `warning`) | | `--min-duplicated-lines` | | Minimum percentage of duplication to report (0-100) | 0 | | `--mcp` | | Serve the [Model Context Protocol over stdio](ai-ready.md#stdio-transport-rust-v5): scan PATHs once, then expose `check_duplication` / `get_statistics` / `check_current_directory` tools to MCP clients | off | | `--summary` | | Print a codebase summary: top files and folders by tokens, lines, size, and a complexity estimate. See [Summary](#summary) | off | | `--summary-top` | | Number of entries in each summary top list | 10 | | `--summary-by` | | Summary sort metric: `tokens`, `lines`, `size`, `complexity` | `tokens` | | `--complexity` | | Print the summary tables ranked by complexity without running clone detection. See [Complexity only](#complexity-only) | off | | `--dashboard` | | Print one screen with the health score, project size, duplication, complexity and dead code. See [Dashboard](#dashboard) | off | | `--health` | | Print only the project health badge: one 0-100 score with a grade. See [Health score](#health-score) | off | | `--health-input` | | JSON file with metrics from other tools (coverage, tests, security) to include in the health score | — | | `--history` | | Duplication trend over git history: scan every commit in RANGE (e.g. `v5.0.0..HEAD`) with the same configuration and print a sparkline and a table. See [History](#history) | — | | `--history-since` | | Select commits since DATE (e.g. `2026-01-01`); alone it walks `HEAD`, with `--history` it bounds the range | — | | `--history-every` | | Keep every Nth commit of the series, counted from the newest | 1 | | `--history-limit` | | Maximum number of commits in the series, sampled evenly with both ends kept | 30 | | `--silent` | `-s` | Suppress console output | off | | `--no-tips` | | Suppress tips and promotional messages. Tips are also skipped automatically when stdout is not a terminal (a pipe, a file, a CI log) or when `CI` or `JSCPD_NO_TIPS` is set; `NO_COLOR` only removes the colours | off | | `--version` | `-V` | Print version | — | | `--help` | `-h` | Print help | — | ### Reporters 15 built-in reporters: | Reporter | Output | |----------|--------| | `console` | Clone list + statistics table (default) | | `console-full` | Clone list with source snippets; with `--blame` shows side-by-side author comparison | | `json` | `report/jscpd-report.json` | | `xml` | `report/jscpd-report.xml` | | `csv` | `report/jscpd-report.csv` | | `html` | `report/jscpd-report.html` | | `markdown` | `report/jscpd-report.md` | | `badge` | `report/jscpd-badge.svg` + `report/jscpd-lines-badge.svg` | | `sarif` | `report/jscpd-report.sarif` (GitHub Code Scanning) | | `codeclimate` (alias `gitlab`) | `report/gl-code-quality-report.json` — CodeClimate issue format, ready for GitLab `artifacts:reports:codequality` | | `openmetrics` | `report/jscpd-metrics.txt` — OpenMetrics text format, ready for GitLab `artifacts:reports:metrics` | | `ai` | Token-efficient output for LLM pipelines | | `xcode` | Xcode-compatible warnings | | `threshold` | Exit 1 if duplication percentage exceeds `--threshold` | | `silent` | No console output | The `xml` report is always well-formed XML 1.0: a `]]>` inside a snippet is split across two CDATA sections, and characters XML cannot represent at all (NUL, ANSI escapes, form feeds and other control bytes, U+FFFE/U+FFFF) are replaced with U+FFFD in snippets and file paths, so `xmllint` and XML parsers accept reports built from files that contain them. See [`fixtures/xml-report-demo`](../fixtures/xml-report-demo/README.md). File reporters write into the `--output` directory (default `report/`) using the `jscpd-report.*` prefix (e.g. `jscpd-report.json`, `jscpd-report.sarif`). ### Summary `--summary` appends a codebase summary to the run output — the statistics jscpd already collects while scanning, aggregated to answer "where should I refactor first": ``` Summary (by tokens; 321 files, 129 folders analyzed) Top files: TOKENS LINES SIZE CX DUP% PATH 2052 363 11.4K 80 0.0 files.ts ... Top folders: FILES TOKENS LINES SIZE CX PATH 8 5264 843 26.5K 15 src/core ... ``` - **Top files** lists the top `--summary-top` files ranked by the `--summary-by` metric. Every row carries all metrics, so re-ranking by another lens is a `--summary-by size` (or `lines`, `complexity`) away. - **Top folders** aggregates files into their direct parent directory (each file counted exactly once; no cumulative ancestor totals). - **CX** is a cyclomatic-complexity estimate computed from the token stream, without parsing: one path per function plus one per branch. For folders it is the per-file mean. - *Branches* are the decision-point tokens `if`, `elif`/`elsif`/`elseif`, `unless`, `for`, `foreach`, `while`, `until`, `case`, `cond`, `when`, `catch`, `rescue`, `except`, `and`, `or`, `andalso`, `orelse`, `&&`, `||`, `?` and `??`, matched case-insensitively so uppercase-keyword languages (SQL, PL/SQL, Fortran, COBOL, BASIC) count too. Some languages add their own: each Rust `match` arm (every `=>`, less one per `match`, since three arms are three paths), Swift's `guard`, Go's `select`. Where `?` also marks an optional — TypeScript, JavaScript, Swift, Kotlin, Groovy, C# — it counts only when it opens a ternary, not in `String?` or `x?.y`. - *Functions* are counted by their declaring keyword (`def`, `fn`, `func`, `function`, `fun`, and `=>` for JavaScript arrow functions) and, in C, C++, Java, Objective-C and C#, by the `head(args) {` shape told apart from an `if (…) {`. A language with neither keeps a baseline of one path per file. - *Not counted:* the body of a `"""` or `'''` string in the languages that have one (Python, Kotlin, Scala, Groovy, Swift, Java, Julia, Elixir, Dart), so a docstring saying "if the value is big" is not three branches; and a keyword-shaped word the file itself binds as a name — `case = 3`, `x.case`, `f(case, when)`. Languages that branch without such tokens (Smalltalk `ifTrue:` messages, Prolog clauses) still read low, so treat CX as a ranking signal first. See [`fixtures/summary-demo`](../fixtures/summary-demo/README.md) for one file per language whose complexity can be counted by hand. - **DUP%** is the share of the file's lines covered by detected clone fragments (both fragments of a clone count toward their files; display is capped at 100%). The summary is fully opt-in and computed after detection from data already in memory, so runs without `--summary` are unaffected. It integrates with: - `console` / `console-full` — the block shown above - `ai` — a compact, LLM-token-efficient variant (one line per file/folder) - `json` — an additive `summary` key in `jscpd-report.json` (absent when the flag is off, so the schema is unchanged for existing consumers) Config file equivalents: `"summary": true`, `"summaryTop": 10`, `"summaryBy": "tokens"`. ```bash # Refactoring hotspots: biggest files by tokens plus duplication share jscpd ./src --summary # Agent-friendly: compact clone list + compact summary jscpd ./src --summary --reporters ai --no-tips # Focus on the most complex files, top 5 lists, machine-readable jscpd ./src --summary --summary-by complexity --summary-top 5 --reporters json ``` ### Complexity only `--complexity` answers the complexity half of the summary without the clone run: files are walked and tokenized with the same filters, complexity is counted, and detection never starts (about 2.5 times faster than a clone run on a 565 MB `node_modules`). The tables are the ones above, ranked by complexity unless `--summary-by` or the `summaryBy` config key names another metric, and without the `DUP%` column. Reporters: `console`, `ai` (compact) and `json`, which writes `jscpd-complexity.json` with the same `summary` object as the clone report. It cannot be combined with `--dead-code` or `--dashboard`. ```bash jscpd ./src --complexity --summary-top 20 jscpd ./src --complexity --reporters ai --no-tips ``` Prose and data files (markdown, reStructuredText, AsciiDoc, text, logs, CSV, JSON, YAML, TOML, INI, properties) have complexity `0` in both `--complexity` and `--summary`: an "if" in a README is a word and `||` in a lock file is a version range. ### Dashboard `--dashboard` prints the whole picture of a project on one screen, under its [health badge](#health-score): its size and largest formats, duplication with the clone count per kind and a breakdown by format, total and mean complexity with the most complex files, and, for JavaScript, TypeScript and Python, dead code by category with the largest findings. `--summary-top N` sets the rows per list (default 5). Detection options (`--min-tokens`, `--ignore-identifiers`, `--kind`, `--ignore`, …) apply as in a normal run, and the dead-code options (`--entry`, `--dead-code-categories`, `--min-confidence`) apply to its dead-code section. Reporters: `console` (the default), `json`, which writes every section of the screen to `jscpd-dashboard.json`, `badge`, which writes `jscpd-health-badge.svg`, and `markdown`/`html`, which write the same sections to `jscpd-dashboard.md`/`jscpd-dashboard.html`. ``` ── Duplication ───────────────────────────────────────────── 1.78% duplicated lines · 1 clone (1 exact) By format: DUP% LINES CLONES FORMAT 45.5 5 1 typescript ``` The format breakdown leaves out prose/data formats (markdown, JSON, YAML, …) and markup formats (HTML, CSS, templates, …): neither counts toward the health score's duplication share, so a row for them here would describe a number the score does not have. The dashboard runs a clone scan and a dead-code scan side by side, so it takes about as long as the slower of the two on a small project and close to their sum on a very large one. `--workers N` is a budget for the whole run: the two scans get half of it each, and `--workers 1` runs them one after the other. The exit gates of a clone run apply: `--threshold` compares the duplication percentage as usual, `--exit-code` is returned when clones were found, and `--fail-on-empty` fails a scan that analyzed nothing. The baseline family does not: `--baseline`, `--baseline-from-ref` and `--update-baseline` need a clone report the dashboard does not write, so they warn and are ignored, and `--fail-on-new-clones` is refused rather than passing silently. Bad dead-code options (`--dead-code-categories`, `--min-confidence`) are refused exactly as in `--dead-code`. See [`fixtures/dashboard-demo`](../fixtures/dashboard-demo/README.md) for a runnable example. ### Health score `--health` prints one number for the state of a codebase, and `--dashboard` shows it on top: ``` Health B 74/100 █████████████████▊░░░░░░ 93 lines of code (XS) duplication 75 █████████░░░ 5.4% in typescript (no text) dead code 72 ████████▋░░░ 14.0% complexity 76 █████████▏░░ 0.0% in complex files ``` How it is calculated: 1. **Three shares of the code lines.** *Duplication* is jscpd's duplication percentage over code files, and does not count a duplicated markup, stylesheet or template block (HTML, CSS, Handlebars, …): a repeating template or style rule is not the maintenance problem repeating programming logic is. Component/script languages such as Vue, Svelte, Astro and GraphQL still count in full — the exclusion follows the clone's own format, not the file it lives in, so a `.svelte` or `.vue` file's style or template block is left out of the duplication share even though the component itself counts. The console line names what it actually measured (`5.4% in typescript`) and, only when the project has files in that category, what it left out (`(no text)`, `(no markup/data)`, …) rather than a fixed disclaimer. *Dead code* is the share of lines nothing runs (JavaScript, TypeScript, Python). *Complexity* is the share of code lines that sit in complex files, those with a complexity of 50 or more — complexity hurts when it piles up, and a mean would hide that. Prose and data files (markdown, JSON, YAML, …) are not the project's code and are left out entirely, so a folder of copied JSON snapshots does not lower the score. Because every dimension is a share, a project is not penalized for being large. 2. **A sub-score per dimension** on a half-life curve, `100 · 2^(−share / halfLife)`: 100 at zero, 50 at one half-life, 25 at two, with no cliff and no dead zone. The half-lives — 8.5% duplication, 7.5% dead code, 50% in complex files — are calibrated on 42 open-source projects so that the median project scores 75 in each dimension. 3. **Size** enters once more: in a small project one finding is a large share, so each share is mixed with 2000 lines of "typical project" before it is scored. At 300 lines that prior dominates; at 50,000 it no longer matters. The report shows both the measured `value` and the `adjusted` one. 4. **One score**: the weighted geometric mean of the sub-scores, so a project that is 40% dead code is not rescued by its low duplication. Grades: `A` from 85, `B` from 70, `C` from 55, `D` from 40, then `E`. A dimension that cannot be measured is left out and named (`dead code n/a`) rather than scored as perfect: dead code is skipped when JavaScript, TypeScript and Python are under 5% of the code, and weighs as much as the share of the code it could read otherwise. Two scores are comparable only when they are built from the same dimensions. **Other tools.** `--health-input FILE` (config key `healthInput`) adds metrics from coverage, test or security tools. A metric is either a ready 0-100 `score`, or a `value` with the `halfLife` that turns it into one; `"direction": "higher"` scores the distance to `max` (default 100): ```json { "metrics": [ { "id": "coverage", "value": 81, "direction": "higher", "halfLife": 40 }, { "id": "security", "score": 100, "weight": 2 } ] } ``` The same object can live under `health` in `.jscpd.json`, together with the tuning of the built-in dimensions: `"health": { "duplication": { "halfLife": 5, "weight": 2 }, "deadCode": { "weight": 0 }, "complexFile": 80 }`. A weight of `0` leaves a dimension out. A metric that cannot be scored, or an unknown key, is an error. Reporters: `console` (the badge), `ai` (one line: `health 74 B (duplication 75, dead-code 72, complexity 76; 93 code lines)`), `json` (`jscpd-health.json`: score, grade, size, and for each dimension its value, adjusted value, lines, half-life, weight and score), `badge` (`jscpd-health-badge.svg`), and `markdown`/`html` (`jscpd-health.md`/`jscpd-health.html`, the same score and dimension table). The exit gates of a clone run apply as they do to the dashboard. See [`fixtures/dashboard-demo`](../fixtures/dashboard-demo/README.md#health) for a runnable example. ### History `--history ` answers "is duplication going up or down?" without a hosted dashboard. Every commit `git log` yields for the range is checked out into a temporary detached worktree (the same machinery as `--baseline-from-ref`), scanned with the run's own configuration, and its totals become one point of a series; the current run is the last point. Commits are scanned one after another, since each scan is already parallel across files. ```bash jscpd src --history v5.0.0..HEAD # every commit in the range jscpd src --history-since 2026-01-01 # every commit since a date, up to HEAD jscpd src --history main --history-since 2026-06-01 --history-every 5 --history-limit 12 ``` The `console` and `console-full` reporters append a block with a bar chart of the duplication percentage (its y-axis spans the series' own min and max, so small drifts stay visible), the table, the change from the previous point (red when duplication rose, green when it fell), the overall trend, and, when `--threshold` is set and the latest value sits below it, the headroom: ``` History (since 2026-01-01: 4 commits + working tree) duplicated lines, % of all lines: min 0.0% max 58.1% now 42.9% 58.1% ┤ ██ │ ██ │ ▇▇ ██ ▇▇ ▇▇ 29.0% ┤ ██ ██ ██ ██ │ ██ ██ ██ ██ │ ██ ██ ██ ██ │ ██ ██ ██ ██ 0.0% ┤ ▁▁ ██ ██ ██ ██ └──────────────── 1 2 3 4 5 # COMMIT DATE FILES LINES CLONES DUP LINES DUP% CHANGE SUBJECT 1 0a3e3d5 2026-08-01 1 11 0 0 0.0% initial helpers 2 6728f91 2026-08-08 2 21 1 9 42.9% +42.9 copy total() into b.js 3 aff51e5 2026-08-15 3 31 2 18 58.1% +15.2 and again into c.js 4 817b39d 2026-08-22 2 21 1 9 42.9% -15.2 b.js imports total() instead 5 working 2026-09-12 2 21 1 9 42.9% = (uncommitted changes) Trend: +42.9 points since 0a3e3d5 (2026-08-01) Threshold 50.0% has 7.1 points of headroom: the series never needed it, tighten it with --threshold 42.9 ``` That last line is the manual form of a ratchet: jscpd reports how far the threshold could be tightened and leaves the change to you. The `json` reporter adds a `history` object (`range`, `threshold`, `points[]` with `commit`, `short`, `date`, `subject`, `sources`, `lines`, `tokens`, `clones`, `duplicatedLines`, `percentage`), and the `ai` reporter prints one compact line per point. Other reporters are unchanged. `--history-every N` keeps every Nth commit counted back from the newest, `--history-limit N` (default 30) samples a longer series evenly while keeping both ends. Paths that did not exist at a commit count as zero. The range needs the commits locally: in a shallow CI checkout fetch them first (`fetch-depth: 0`). Config keys: `history`, `historySince`, `historyEvery`, `historyLimit`. See [`fixtures/history-demo`](../fixtures/history-demo/README.md) for a runnable example. ### Baseline Gate CI on *new* duplication only, tolerating clones that already exist. A baseline file records a content-hash fingerprint per accepted clone (the same hash the SARIF reporter emits as `partialFingerprints["jscpdCloneHash/v1"]`); clones absent from it are reported as new. ```bash # Create or refresh the committed baseline jscpd --baseline .jscpd-baseline.json --update-baseline . # Fail when new clones appear (independent of --threshold) jscpd --baseline .jscpd-baseline.json --fail-on-new-clones . # Allow up to 3 new clones jscpd --baseline .jscpd-baseline.json --fail-on-new-clones 3 . # Stateless variant for PR gates: compare against a git ref instead of a file jscpd --baseline-from-ref origin/main --fail-on-new-clones . ``` `--baseline-from-ref` checks the base ref's tree out into a temporary detached worktree, scans it with the same configuration, and compares fingerprints in memory. It costs a second scan of the corpus; the committed file needs only one. In CI, fetch the ref first (`fetch-depth: 0` or `git fetch origin main`). New-clone information flows through the reporters: `[NEW]` markers in `console`/`console-full`, per-clone `isNew` plus `newClones` / `newDuplicatedLines` statistics in `json`, level `error` in `sarif`, severity `major` in `codeclimate`, and `jscpd_new_clones` / `jscpd_new_duplicated_lines` gauges in `openmetrics`. Config file keys: `baseline`, `baselineFromRef`, `failOnNewClones`. ### Blame Output With `--blame --reporters console-full`, clones are displayed with a side-by-side author comparison: ``` 176 │ Andrii Kucherenko │ <= │ 196 │ Josh Soref │ ## TODO 177 │ Andrii Kucherenko │ <= │ 197 │ Josh Soref │ 180 │ Andrii Kucherenko │ == │ 200 │ Andrii Kucherenko │ ## License ``` `==` means both lines were written by the same author; `<=` means different authors (potential copy). ### Exit codes | Code | When | |------|------| | 0 | The scan ran and no gate fired; clones may still have been found and reported | | 1 | `--threshold` exceeded; `--fail-on-new-clones` exceeded; `--fail-on-empty` and no file was analyzed; a reporter failed to write its output; a scan path does not exist; `--format` names a format that is not supported; an invalid flag combination or an unreadable `--config` file | | N | `--exit-code N` (default 1) when at least one clone was found | | 2 | Command-line parse errors: an unknown flag or a missing value | A scan that analyzes no files, because the paths exist but nothing matched the `--format`, `--ignore` and `--pattern` filters or every file was below `--min-tokens`, prints `Warning: jscpd analyzed no files` and still exits 0, so an intentionally empty tree does not break a pipeline. `--fail-on-empty` (config key `failOnEmpty`) turns that into an error for CI jobs where an empty result means a misconfigured scan. Reports are written before the check, so the empty report is still there to inspect. Unknown reporter names remain a warning. See [`fixtures/fail-on-empty-demo`](../fixtures/fail-on-empty-demo/README.md) for a runnable example. ### Examples ```bash # Scan a directory jscpd /path/to/source # or jscpd /path/to/source # Tune sensitivity and pick reporters jscpd /path/to/source --min-tokens 30 --min-lines 3 --reporters console,json,html # Git blame with side-by-side author comparison jscpd /path/to/source --blame --reporters console-full # List supported formats jscpd --list # Use multiple reporters with custom output jscpd ./src -r console,json,sarif -o ./reports # Skip clones within the same directory jscpd --skip-local /path/to/source # Monorepo: don't compare team-owned packages with each other jscpd . --skip-isolated "packages/team-a|packages/team-b" ``` ### Config File Options can also come from a `.jscpd.json` config file (camelCase keys; existing v4 config files work unchanged): ```json { "path": ["./src"], "reporters": ["console", "json"], "minLines": 5, "minTokens": 50, "threshold": 0, "format": ["javascript", "typescript"], "ignore": ["**/node_modules/**"], "ignorePattern": ["generated by .*"], "ignoreIdentifiers": false, "ignoreLiterals": false, "ignoreAnnotations": false, "maxGapLines": 0, "gitignore": true, "mode": "mild" } ``` Isolation groups use the nested-array form in the config file: `"skipIsolated": [["packages/a", "packages/b"]]`. Config discovery order: `--config ` → `.jscpd.json` → `.config/jscpd.json` (the [dot-config convention](https://dot-config.github.io/), also accepts `.config/.jscpd.json`) → the `jscpd` key in `package.json`. ### Ignoring source regions Use `--ignore-pattern` (or `ignorePattern` in `.jscpd.json`) when only part of a file should be excluded. Each regular expression is matched against the raw source text before tokenization, and tokens that overlap a match are omitted from clone detection. This matches the v4 behavior; v5 uses [Rust regex syntax](https://docs.rs/regex/latest/regex/#syntax), which does not support look-around or backreferences. A pattern that fails to compile is skipped with a warning. The CLI flag splits its value on commas, so a regular expression that itself contains a comma (a `{1,3}` repetition, a character class such as `[,;]`) must be set in the config file instead. For a one-off region, place `jscpd:ignore-start` and `jscpd:ignore-end` in comments that are valid for the source language: ```javascript // jscpd:ignore-start const generatedLookup = { alpha: 1, beta: 2, }; // jscpd:ignore-end ``` All source between the markers is excluded during tokenization. This works in every detection mode. License headers are a common use case in languages whose comments are tokenized: C#, Java, Go, Python and every other format handled by the generic tokenizer. JavaScript and TypeScript comments never produce tokens, so a header in those files is never part of a clone and needs no exception. Mark a header explicitly when only a few files need it: ```csharp // jscpd:ignore-start /* * Copyright 2026 Example Authors * Licensed under the Example License */ // jscpd:ignore-end ``` For the same block-comment header across many files, configure one anchored, dot-all regular expression instead: ```json { "ignorePattern": ["(?s)\\A/\\*.*?\\*/"] } ``` The JSON escaping above produces the Rust regular expression `(?s)\A/\*.*?\*/`, which matches the first block comment only when it starts at the beginning of the file. Adjust the expression to the comment style and exact license text used by the project. `--mode weak` is a blunter alternative: it drops every comment from detection in every language, so duplicated comments elsewhere in the code are ignored too. ### Type-2 clones: renamed identifiers, literals and annotations By default two blocks are a clone only when their tokens are identical. Three opt-in flags normalize token classes before hashing so that blocks which differ only in names or values match as well (Type-2 clones, the same idea as PMD CPD's `--ignore-identifiers` / `--ignore-literals` / `--ignore-annotations`): | Flag | Config key | What is normalized | |------|------------|--------------------| | `--ignore-identifiers` | `ignoreIdentifiers` | Every identifier hashes as `$id`. Keywords are kept, so `if` never matches `while`: the JavaScript/TypeScript tokenizer classifies keywords itself, other languages use a built-in list of common keywords | | `--ignore-literals` | `ignoreLiterals` | String literals hash as `$str`, numeric literals as `$num`; `true`, `null` and regex literals keep their value | | `--ignore-annotations` | `ignoreAnnotations` | `@Name`, `@a.b.Name` and `@Name(...)` runs are dropped in Java, Kotlin, Scala, Groovy, Python, Dart, Swift, JavaScript and TypeScript. Formats where `@` prefixes a variable or directive (Ruby, Perl, Razor, T-SQL, CSS) are untouched | Positions in reports still point at the original source. Each clone carries a `kind`: `exact` when the raw tokens of both fragments are identical, `renamed` when they match only after normalization. The console prints `Clone found (javascript, renamed)`, the `ai` reporter appends `(renamed)`, the JSON report adds `"kind": "renamed"` to each duplicate, SARIF reports renamed clones under the rule `jscpd/renamed-code` (exact ones stay `jscpd/duplicate-code`), and Code Climate uses the same two `check_name` values. A run without these flags reports only `exact` clones and its output is unchanged. When `--max-gap-lines` merges two renamed halves, the result is `similar`: a merged clone is no longer identical even after normalization, so `similar` takes precedence over `renamed`. ```bash jscpd --ignore-identifiers src/ # function a(x) {…} matches function b(y) {…} jscpd --ignore-identifiers --ignore-literals src/ # …and 10 matches 25, 'dev' matches 'prod' jscpd --ignore-annotations src/main/java/ # @Override / @Deprecated no longer split a clone ``` Normalized runs find more and longer clones than exact runs, so their snippet fingerprints (`jscpdCloneHash/v1`) differ from an exact run's. Keep a separate `--baseline` file per configuration. See [`fixtures/type2-demo`](../fixtures/type2-demo/README.md) for a runnable example of each flag. ### Type-3 clones: near-miss merging with `--max-gap-lines` A copy with a line inserted, removed or changed in the middle shows up as two shorter exact clones with a gap between them, each of which must clear `--min-tokens` and `--min-lines` on its own. `--max-gap-lines N` (config key `maxGapLines`) merges clones of the same file pair whose fragments follow each other in *both* files with at most `N` unmatched lines in between, and reports the result as one clone of kind `similar`: ```bash jscpd src/ # a.js 1-4 ↔ b.js 1-4, a.js 4-9 ↔ b.js 5-10: two exact clones jscpd --max-gap-lines 1 src/ # Clone found (javascript, similar (gap) ~0.85): a.js 1-9 ↔ b.js 1-10 ``` A merged clone's `tokens` is the number of matched tokens and `similarity` is that number divided by the tokens of the longer merged span, so a single inserted line in a 60-token block gives roughly `0.9`. Chains of matches merge transitively, and the merge is applied after every other filter (`--min-lines`, `--skip-local`, `--skip-isolated`). Because merging only ever joins clones the exact run already reported, it cannot introduce a match that was not there; it removes fragmentation. It applies to every language. Reporting: the console prints `Clone found (javascript, similar (gap) ~0.85)`, the `ai` reporter appends `[~0.85 gap]`, the JSON report adds `"kind": "similar"`, a `"similarity"` value and `"method": "gap"`, SARIF files merged clones under `jscpd/similar-code` with `similarity` and `similarity_method` properties, and Code Climate uses the same `check_name`. The method is shown because the two near-miss mechanisms score on different scales: `gap` is matched tokens over the merged span, `ast` (from `--similarity`, below) is structural overlap of whole functions. A merge is refused when its similarity would fall below `0.5`, that is when the gap holds more tokens than the halves match (one very long inserted line, say); the halves are then reported separately as before. Duplicated-line statistics count only the matched lines of a merged clone, so enabling the merge does not move `--threshold`. With the default `0` the merge pass is skipped entirely and output is identical to earlier releases. See [`fixtures/type3-demo`](../fixtures/type3-demo/README.md) for a runnable example. ### Filtering by kind with `--kind` `--kind` (config key `kind`) keeps only the clones of the kinds it lists: `exact`, `renamed`, `similar`, or one of the two mechanisms behind `similar`, `gap` (`--max-gap-lines`) and `ast` (`--similarity`). Statistics, `--threshold` and every reporter see the filtered list. The filter never switches a detector on: `--kind ast` without `--similarity` warns that no such clones can be found, and an unknown kind is an error, so a typo cannot turn a scan silently clean. ```bash jscpd --ignore-identifiers --kind renamed src/ # only the renamed copies jscpd --max-gap-lines 2 --similarity 0.8 --kind gap,ast src/ # only near-miss clones ``` See [`fixtures/type3-demo`](../fixtures/type3-demo/README.md#keeping-one-kind---kind) for a runnable example. ### Function-level similarity with `--similarity` Edits spread through a function rather than concentrated in one gap still escape a token window. `--similarity RATIO` (config key `similarity`, a number in `(0, 1]`; the default `1` means exact matches only, so the pass never runs until you set a lower value) compares whole functions instead: every function declaration, function expression, method and arrow function in a JavaScript, TypeScript, JSX or TSX file is summarized by the bag of 4-grams over the pre-order sequence of its syntax-tree node *types*, and two functions are reported as one `similar` clone when the weighted Jaccard index of their bags reaches `RATIO`. Names and literal values are not part of the summary, so a renamed copy scores `1.0`; one inserted line scores about `0.9`; two inserted statements plus renames score about `0.75`. Candidates come from a MinHash index, so the search stays close to linear in the number of functions. ```bash jscpd --similarity 0.85 src/ # near-identical structure: renames, literal changes, a one-line edit jscpd --similarity 0.7 src/ # looser: a couple of added or removed statements ``` Functions must clear `--min-tokens` and `--min-lines` on their own, nested functions are never paired with their parent, and a pair that an exact, renamed or merged clone already covers is not reported again. Reporting is the same as for merged clones except for the method: the console prints `Clone found (javascript, similar (ast) ~0.75)`, the `ai` reporter `[~0.75 ast]`, JSON carries `"method": "ast"` and SARIF `similarity_method`; `tokens` is the smaller function's token count and the fragments span the whole functions. Values outside `(0, 1]` print a warning and fall back to `1`. Scoring needs a syntax tree, and today only JavaScript/TypeScript have one (oxc). Each language plugs in through the `FunctionExtractor` trait in `cpd-tokenizer` (`functions.rs`): a grammar id, the formats it serves, and a walk that opens a function at every function-like node and records the node-type sequence inside it. Signatures carry their grammar id and are only compared within one grammar, so a tree-sitter-backed extractor for another language is a self-contained addition; the scoring, CLI, MCP tool and reporters need no change. Formats without an extractor are a silent no-op. The MCP `check_duplication` tool accepts the same `similarity` argument and returns the structurally similar project functions for each function in the snippet. ## How detection works jscpd is a token-based detector, but the tokens come from each language's own syntax rather than from splitting text on whitespace. A scan goes through these stages: 1. **Format detection.** The file extension (or a name such as `Makefile` / `Dockerfile`, see `--formats-names`) selects one of the 224 formats, and the format selects the tokenizer. 2. **Tokenization.** Every format has its comment style (`//` and `/* */`, `#`, `--`, Lua's `--[[ ]]`, `;`, Visual Basic's `'`, none for Markdown) and string-literal rules, so a comment marker inside a string, or a `/*` in Markdown prose, does not swallow the rest of the file. Each token is classified as identifier, literal, punctuation, comment or whitespace. - **JavaScript, TypeScript, JSX, TSX** are tokenized by the [oxc](https://oxc.rs) parser: template literals, regular expressions, JSX and decorators come out as the tokens the language defines, and a recoverable parse error does not change the stream. With `--cross-formats`, erasable TypeScript-only syntax (type annotations, interfaces, type aliases, generics, access modifiers) is removed via the syntax tree so the remaining tokens match the equivalent JavaScript; constructs with runtime meaning (`enum`, non-declare `namespace`, parameter properties) are kept. - **Vue, Svelte, Astro** files are split into `