skills/kucherenko__jscpd
1
0
Fork 0
mirror of https://github.com/kucherenko/jscpd.git synced 2026-09-19 08:11:03 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(cli): --history, duplication trend over git history

Browse Source 
Closes #1002 (the --history half; --ratchet is deliberately not
implemented, the report shows the headroom instead).

`jscpd src --history v5.0.0..HEAD` (or `--history-since 2026-01-01`)
lists the commits with `git log`, checks each one out into a temporary
detached worktree (the --baseline-from-ref machinery, now shared through
baseline_ref::map_scan_paths), scans it with the run's own configuration
and keeps the totals; the current run is the series' last point.
`--history-every N` keeps every Nth commit counted from the newest,
`--history-limit N` (default 30) samples a long series evenly keeping
both ends. Nothing runs unless one of the flags is given.

Output: the console reporter appends a sparkline of the duplication
percentage, a table (commit, date, files, lines, clones, dup lines,
dup%, change, subject) with the change column red when duplication rose
and green when it fell, the overall trend line, and, when --threshold is
set and the latest value is under it, the headroom with the value the
threshold could be tightened to. The json reporter adds a `history`
object; the ai reporter prints one compact line per point. Other
reporters are unchanged.

New cpd_core::history (model, sparkline, change/headroom helpers with
unit tests), cpd::history (git log parsing, sampling, worktree scans),
cpd_reporter::history_render. Config keys history / historySince /
historyEvery / historyLimit; action input `history`. Integration tests
build a four-commit repo with fixed dates and check the JSON series,
the console block, every/limit thinning and the error paths. 969
workspace tests pass, clippy clean.

Docs: History section and option rows in docs/rust.md, README bullet,
action input row, fixtures/history-demo/README.md with a script that
builds the demo repository and the expected output.

Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
This commit is contained in:
apk
2026-09-12 07:50:18 +02:00
parent eebc1cd7bd
commit adbf5ecf3d
27 changed files with 1206 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1
View File
@@ -85,6 +85,7 @@ jscpd v5 is a Rust engine that ships as a self-contained binary — no runtime r
- **Exit codes you can gate on** — an unknown `--format`, a missing scan path and a reporter that cannot write its file exit 1 instead of passing with an empty report; `--fail-on-empty` fails a scan that analyzed no files (see [Exit codes](docs/rust.md#exit-codes))
- **GitLab-ready reporters** — `codeclimate` (`gl-code-quality-report.json`) and `openmetrics` (`jscpd-metrics.txt`) plug into `artifacts:reports`
- **Git blame** with side-by-side author comparison (`--blame --reporters console-full`)
- **`--history`** — duplication trend over git history: `jscpd src --history v5.0.0..HEAD` scans every commit in the range and prints a sparkline, a per-commit table with the change between points, the overall trend, and how far `--threshold` could be tightened (see [docs](docs/rust.md#history))
- **`--summary`** — codebase summary: top files and folders by tokens, lines, size, and a complexity estimate — refactoring hotspots straight from the scan (see [docs](docs/rust.md#summary))
- **`--mcp`** — built-in MCP server over stdio with fully described tools: point your AI assistant at the binary and it can check snippets for duplication against your codebase, or find structurally similar functions with a `similarity` argument (see [docs](docs/ai-ready.md#stdio-transport-rust-v5))
- **AI reporter** — token-efficient output for LLM pipelines (~79% fewer tokens than console)
action.yml
+8
View File
@@ -73,6 +73,10 @@ inputs:
description: "Compare against an ephemeral baseline built from a git ref's tree (e.g. origin/main); needs the ref fetched (fetch-depth: 0). Conflicts with baseline"
required: false
default: ""
history:
description: "Duplication trend over git history: scan every commit in this range (e.g. v5.0.0..HEAD) and print a chart and table in the log; needs the range fetched (fetch-depth: 0)"
required: false
default: ""
blame:
description: "Enrich clones with git blame data"
required: false
@@ -286,6 +290,7 @@ runs:
INPUT_FAIL_ON_NEW_CLONES: ${{ inputs.fail-on-new-clones }}
INPUT_FAIL_ON_EMPTY: ${{ inputs.fail-on-empty }}
INPUT_BASELINE_FROM_REF: ${{ inputs.baseline-from-ref }}
INPUT_HISTORY: ${{ inputs.history }}
INPUT_BLAME: ${{ inputs.blame }}
INPUT_EXIT_CODE: ${{ inputs.exit-code }}
INPUT_PATTERN: ${{ inputs.pattern }}
@@ -374,6 +379,9 @@ runs:
elif [ -n "$INPUT_BASELINE_FROM_REF" ]; then
ARGS+=("--baseline-from-ref" "$INPUT_BASELINE_FROM_REF")
fi
if [ -n "$INPUT_HISTORY" ]; then
ARGS+=("--history" "$INPUT_HISTORY")
fi
# Fail on new clones: empty=omit, 'true'=bare flag (N=0), integer=value
if [ -n "$INPUT_FAIL_ON_NEW_CLONES" ]; then
if [ "$INPUT_FAIL_ON_NEW_CLONES" = "true" ]; then
docs/ci-and-hooks.md
+1
View File
@@ -57,6 +57,7 @@ The workflow fails if more than 5% of the code is duplicated.
| `update-baseline` | Rewrite the baseline file from the current run (requires `baseline`) | `false` |
| `fail-on-new-clones` | Exit 1 on new clones (`true`, or an integer N to allow up to N) | — |
| `fail-on-empty` | Exit 1 when the scan analyzes no files (paths exist but nothing matched the filters) | `false` |
| `history` | Duplication trend over git history: scan every commit in this range (e.g. `v5.0.0..HEAD`) and print a chart and table in the log (needs `fetch-depth: 0`) | — |
| `baseline-from-ref` | Compare against an ephemeral baseline built from a git ref (needs `fetch-depth: 0`) | — |
| `blame` | Enrich clones with git blame data | `false` |
| `exit-code` | Exit with code when duplicates found (`true` or integer) | — |
docs/rust.md
+33
View File
@@ -106,6 +106,10 @@ cpd [OPTIONS] [PATH]...
| `--summary` | | Print a codebase summary: top files and folders by tokens, lines, size, and a complexity estimate. See [Summary](#summary) | off |
| `--summary-top` | | Number of entries in each summary top list | 10 |
| `--summary-by` | | Summary sort metric: `tokens`, `lines`, `size`, `complexity` | `tokens` |
| `--history` | | Duplication trend over git history: scan every commit in RANGE (e.g. `v5.0.0..HEAD`) with the same configuration and print a sparkline and a table. See [History](#history) | — |
| `--history-since` | | Select commits since DATE (e.g. `2026-01-01`); alone it walks `HEAD`, with `--history` it bounds the range | — |
| `--history-every` | | Keep every Nth commit of the series, counted from the newest | 1 |
| `--history-limit` | | Maximum number of commits in the series, sampled evenly with both ends kept | 30 |
| `--silent` | `-s` | Suppress console output | off |
| `--no-tips` | | Suppress tips and promotional messages. Tips are also skipped automatically when stdout is not a terminal (a pipe, a file, a CI log) or when `CI` or `JSCPD_NO_TIPS` is set; `NO_COLOR` only removes the colours | off |
| `--version` | `-V` | Print version | — |
@@ -175,6 +179,35 @@ cpd ./src --summary --reporters ai --no-tips
cpd ./src --summary --summary-by complexity --summary-top 5 --reporters json
```
### History
`--history <range>` answers "is duplication going up or down?" without a hosted dashboard. Every commit `git log` yields for the range is checked out into a temporary detached worktree (the same machinery as `--baseline-from-ref`), scanned with the run's own configuration, and its totals become one point of a series; the current run is the last point. Commits are scanned one after another, since each scan is already parallel across files.
```bash
jscpd src --history v5.0.0..HEAD # every commit in the range
jscpd src --history-since 2026-01-01 # every commit since a date, up to HEAD
jscpd src --history main --history-since 2026-06-01 --history-every 5 --history-limit 12
```
The console reporter appends a block with a sparkline of the duplication percentage, the table, the change from the previous point (red when duplication rose, green when it fell), the overall trend, and, when `--threshold` is set and the latest value sits below it, the headroom:
```
History (since 2026-01-01: 4 commits + working tree)
▁▆█▆▆ min 0.0% max 58.1% now 42.9%
COMMIT DATE FILES LINES CLONES DUP LINES DUP% CHANGE SUBJECT
0a3e3d5 2026-08-01 1 11 0 0 0.0% initial helpers
6728f91 2026-08-08 2 21 1 9 42.9% +42.9 copy total() into b.js
aff51e5 2026-08-15 3 31 2 18 58.1% +15.2 and again into c.js
817b39d 2026-08-22 2 21 1 9 42.9% -15.2 b.js imports total() instead
working 2026-09-12 2 21 1 9 42.9% = (uncommitted changes)
Trend: +42.9 points since 0a3e3d5 (2026-08-01)
Threshold 50.0% has 7.1 points of headroom: the series never needed it, tighten it with --threshold 42.9
```
That last line is the manual form of a ratchet: jscpd reports how far the threshold could be tightened and leaves the change to you. The `json` reporter adds a `history` object (`range`, `threshold`, `points[]` with `commit`, `short`, `date`, `subject`, `sources`, `lines`, `tokens`, `clones`, `duplicatedLines`, `percentage`), and the `ai` reporter prints one compact line per point. Other reporters are unchanged.
`--history-every N` keeps every Nth commit counted back from the newest, `--history-limit N` (default 30) samples a longer series evenly while keeping both ends. Paths that did not exist at a commit count as zero. The range needs the commits locally: in a shallow CI checkout fetch them first (`fetch-depth: 0`). Config keys: `history`, `historySince`, `historyEvery`, `historyLimit`. See [`fixtures/history-demo`](../fixtures/history-demo/README.md) for a runnable example.
### Baseline
Gate CI on *new* duplication only, tolerating clones that already exist. A baseline file records a content-hash fingerprint per accepted clone (the same hash the SARIF reporter emits as `partialFingerprints["jscpdCloneHash/v1"]`); clones absent from it are reported as new.
fixtures/history-demo/README.md
+80
View File
@@ -0,0 +1,80 @@
# history demo
`--history` scans every commit in a git range with the run's own configuration
and prints the duplication trend: a sparkline, one table row per commit, the
change between points, and, with `--threshold`, how far the threshold could be
tightened. A trend needs commits, so this demo builds its own repository in a
temporary directory instead of shipping files; run it from the repository root
at default thresholds.
| Step | Commit | Files | Clones |
|------|--------|-------|--------|
| 1 | `initial helpers` | `a.js` | 0 |
| 2 | `copy total() into b.js` | `a.js`, `b.js` | 1 |
| 3 | `and again into c.js` | `a.js`, `b.js`, `c.js` | 2 |
| 4 | `b.js imports total() instead` | `b.js` rewritten | 1 |
## Build the repository
```bash
demo=$(mktemp -d) && cd "$demo" && git init -q && mkdir src
git config user.email demo@example.com && git config user.name demo && git config commit.gpgsign false
fn() { printf 'export function %s(a, b, c) {\n const first = a * b + c;\n const second = first - a / b;\n const third = second + c * c;\n const fourth = third - first + a;\n const fifth = fourth * second - b;\n console.log(first, second, third, fourth, fifth);\n return [first, second, third, fourth, fifth];\n}\n' "$1"; }
one() { printf 'export const %s = (n) => n * %d + %d;\n' "$1" "$2" "$3"; }
snap() { git add -A && GIT_COMMITTER_DATE="$2" git commit -q -m "$1" --date="$2"; }
{ fn total; one tax 3 1; one fee 5 2; } > src/a.js && snap 'initial helpers' 2026-08-01T10:00:00
{ fn total; one rate 7 3; } > src/b.js && snap 'copy total() into b.js' 2026-08-08T10:00:00
{ fn total; one discount 9 4; } > src/c.js && snap 'and again into c.js' 2026-08-15T10:00:00
{ one rate 7 3; one rate2 11 5; } > src/b.js && snap 'b.js imports total() instead' 2026-08-22T10:00:00
```
## Trend over every commit
```bash
jscpd src --history-since 2026-01-01 --no-colors
# Found 1 clones.
#
# History (since 2026-01-01: 4 commits + working tree)
# ▁▆█▆▆ min 0.0% max 58.1% now 42.9%
# COMMIT DATE FILES LINES CLONES DUP LINES DUP% CHANGE SUBJECT
# <sha> 2026-08-01 1 11 0 0 0.0% initial helpers
# <sha> 2026-08-08 2 21 1 9 42.9% +42.9 copy total() into b.js
# <sha> 2026-08-15 3 31 2 18 58.1% +15.2 and again into c.js
# <sha> 2026-08-22 2 21 1 9 42.9% -15.2 b.js imports total() instead
# working <today> 2 21 1 9 42.9% = (uncommitted changes)
# Trend: +42.9 points since <sha> (2026-08-01)
```
Commit hashes differ per machine because the author date is fixed but the
author is yours; everything else is identical. With colors on, `+` changes are
red and `-` changes green.
## Threshold headroom instead of an automatic ratchet
```bash
jscpd src --history HEAD~3..HEAD --threshold 50 --no-colors
# History (HEAD~3..HEAD: 3 commits + working tree)
# ...
# Threshold 50.0% has 7.1 points of headroom: the series never needed it, tighten it with --threshold 42.9
```
When the latest value is over the threshold the usual threshold error fires
instead and the run exits 1.
## Thinning a long series
```bash
jscpd src --history-since 2026-01-01 --history-every 2 --no-colors # every 2nd commit, newest kept
# History (since 2026-01-01: 2 commits + working tree)
jscpd src --history-since 2026-01-01 --history-limit 2 --no-colors # first and last only
# History (since 2026-01-01: 2 commits + working tree)
```
## JSON
```bash
jscpd src --history-since 2026-01-01 --reporters json --output report
# report/jscpd-report.json gains "history": { "range", "threshold", "points": [...] }
```
Clean up with `cd - && rm -rf "$demo"`.
rust/crates/cpd-core/src/history.rs
+207
View File
@@ -0,0 +1,207 @@
//! Duplication trend over git history (`--history`, issue #1002).
//!
//! One [`HistoryPoint`] per scanned commit, oldest first, plus a final point
//! for the working tree. The CLI collects the points by scanning each commit
//! in a temporary worktree with the run's own configuration; this module only
//! holds the data model and the pure helpers reporters need (sparkline,
//! per-point change, threshold hint). Nothing here touches git.
use serde::{Deserialize, Serialize};
/// Identifier used for the working-tree point instead of a commit hash.
pub const WORKING_TREE: &str = "working tree";
/// Detection totals for one commit (or the working tree).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct HistoryPoint {
/// Full commit hash, or [`WORKING_TREE`] for the uncommitted state.
pub commit: String,
/// Abbreviated hash for display (7 characters), or `working`.
pub short: String,
/// Committer date as `YYYY-MM-DD`; the detection date for the working tree.
pub date: String,
/// First line of the commit message; empty for the working tree.
pub subject: String,
pub sources: u64,
pub lines: u64,
pub tokens: u64,
pub clones: u64,
pub duplicated_lines: u64,
/// Duplicated lines as a percentage of all lines.
pub percentage: f64,
}
impl HistoryPoint {
pub fn is_working_tree(&self) -> bool {
self.commit == WORKING_TREE
}
}
/// The series a `--history` run produces.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct History {
/// What was walked, for display: `v5.0.0..HEAD`, `since 2026-01-01`.
pub range: String,
/// `--threshold` in effect, if any; drives the tightening hint.
pub threshold: Option<f64>,
/// Oldest first; the last point is the working tree.
pub points: Vec<HistoryPoint>,
}
/// Levels used by [`sparkline`], lowest to highest.
const BARS: [char; 8] = ['▁', '▂', '▃', '▄', '▅', '▆', '▇', '█'];
/// One character per value, scaled between the series' min and max. A flat
/// series renders at mid height so it still reads as "present, unchanged".
pub fn sparkline(values: &[f64]) -> String {
let (min, max) = values
.iter()
.fold((f64::INFINITY, f64::NEG_INFINITY), |(lo, hi), v| {
(lo.min(*v), hi.max(*v))
});
values
.iter()
.map(|v| {
if max <= min {
BARS[3]
} else {
let level = ((v - min) / (max - min) * (BARS.len() - 1) as f64).round() as usize;
BARS[level.min(BARS.len() - 1)]
}
})
.collect()
}
impl History {
/// Percentages in series order.
pub fn percentages(&self) -> Vec<f64> {
self.points.iter().map(|p| p.percentage).collect()
}
pub fn sparkline(&self) -> String {
sparkline(&self.percentages())
}
/// Change in percentage points from the previous point; `None` for the
/// first one.
pub fn change_at(&self, index: usize) -> Option<f64> {
if index == 0 || index >= self.points.len() {
return None;
}
Some(self.points[index].percentage - self.points[index - 1].percentage)
}
/// Change in percentage points from the first to the last point.
pub fn overall_change(&self) -> Option<f64> {
match (self.points.first(), self.points.last()) {
(Some(first), Some(last)) if self.points.len() > 1 => {
Some(last.percentage - first.percentage)
}
_ => None,
}
}
/// Room between the threshold and the latest value, in percentage points,
/// when the latest value is below the threshold. This is what `--ratchet`
/// would apply automatically; jscpd only reports it.
pub fn threshold_headroom(&self) -> Option<f64> {
let threshold = self.threshold?;
let last = self.points.last()?;
let headroom = threshold - last.percentage;
(headroom > 0.05).then_some(headroom)
}
pub fn commit_count(&self) -> usize {
self.points.iter().filter(|p| !p.is_working_tree()).count()
}
}
#[cfg(test)]
mod tests {
use super::*;
fn point(short: &str, percentage: f64) -> HistoryPoint {
HistoryPoint {
commit: if short == "working" {
WORKING_TREE.to_string()
} else {
format!("{short}0000000000000000000000000000000000")
},
short: short.to_string(),
date: "2026-09-12".to_string(),
subject: String::new(),
sources: 10,
lines: 1000,
tokens: 5000,
clones: 2,
duplicated_lines: (percentage * 10.0) as u64,
percentage,
}
}
fn history(values: &[f64], threshold: Option<f64>) -> History {
History {
range: "a..b".to_string(),
threshold,
points: values
.iter()
.enumerate()
.map(|(i, v)| point(&format!("c{i}"), *v))
.collect(),
}
}
#[test]
fn sparkline_scales_between_min_and_max() {
assert_eq!(sparkline(&[0.0, 50.0, 100.0]), "▁▅█");
assert_eq!(sparkline(&[1.0, 1.0, 1.0]), "▄▄▄");
assert_eq!(sparkline(&[]), "");
}
#[test]
fn change_at_is_difference_to_previous_point() {
let h = history(&[2.0, 3.5, 3.0], None);
assert_eq!(h.change_at(0), None);
assert!((h.change_at(1).unwrap() - 1.5).abs() < 1e-9);
assert!((h.change_at(2).unwrap() + 0.5).abs() < 1e-9);
assert_eq!(h.change_at(3), None);
}
#[test]
fn overall_change_spans_first_to_last() {
assert!((history(&[4.0, 1.0, 2.5], None).overall_change().unwrap() + 1.5).abs() < 1e-9);
assert_eq!(history(&[4.0], None).overall_change(), None);
}
#[test]
fn threshold_headroom_only_when_below_threshold() {
assert!(
(history(&[3.0, 2.1], Some(5.0))
.threshold_headroom()
.unwrap()
- 2.9)
.abs()
< 1e-9
);
assert_eq!(history(&[3.0, 6.0], Some(5.0)).threshold_headroom(), None);
assert_eq!(history(&[3.0, 5.0], Some(5.0)).threshold_headroom(), None);
assert_eq!(history(&[3.0, 2.0], None).threshold_headroom(), None);
}
#[test]
fn commit_count_excludes_working_tree() {
let mut h = history(&[1.0, 2.0], None);
h.points.push(point("working", 2.0));
assert_eq!(h.commit_count(), 2);
assert!(h.points[2].is_working_tree());
}
#[test]
fn json_uses_camel_case() {
let json = serde_json::to_string(&history(&[1.5], Some(3.0))).unwrap();
assert!(json.contains("\"duplicatedLines\""));
assert!(json.contains("\"threshold\":3.0"));
}
}
rust/crates/cpd-core/src/lib.rs
+1
View File
@@ -1,5 +1,6 @@
pub mod detect;
pub mod hash;
pub mod history;
pub mod models;
pub mod paths;
pub mod similarity;
rust/crates/cpd-reporter/src/ai.rs
+3
View File
@@ -92,6 +92,9 @@ impl Reporter for AiReporter {
println!("---");
crate::summary_render::print_summary_compact(summary);
}
if let Some(history) = ctx.history {
crate::history_render::print_history_compact(history);
}
Ok(())
}
}
rust/crates/cpd-reporter/src/badge.rs
+2
View File
@@ -120,6 +120,7 @@ mod tests {
stats: &stats_with_pct(5.0, 10),
duration: Duration::ZERO,
summary: None,
history: None,
};
reporter.report(&[], &ctx, &dir).unwrap();
let content = std::fs::read_to_string(dir.join("jscpd-badge.svg")).unwrap();
@@ -138,6 +139,7 @@ mod tests {
stats: &stats_with_pct(pct, duplicated_lines),
duration: Duration::ZERO,
summary: None,
history: None,
};
reporter.report(&[], &ctx, &dir).unwrap();
(dir.clone(), dir.join("jscpd-badge.svg"))
rust/crates/cpd-reporter/src/codeclimate.rs
+1
View File
@@ -182,6 +182,7 @@ mod tests {
stats: &stats,
duration: Duration::ZERO,
summary: None,
history: None,
};
reporter.report(clones, &ctx, &dir).unwrap();
let content = std::fs::read_to_string(dir.join("gl-code-quality-report.json")).unwrap();
rust/crates/cpd-reporter/src/console.rs
+4
View File
@@ -58,6 +58,9 @@ impl Reporter for ConsoleReporter {
if let Some(summary) = ctx.summary {
crate::summary_render::print_summary(summary, &self.style);
}
if let Some(history) = ctx.history {
crate::history_render::print_history(history, &self.style);
}
Ok(())
}
}
@@ -82,6 +85,7 @@ mod tests {
stats: &one_clone_stats(),
duration: Duration::ZERO,
summary: None,
history: None,
};
assert!(
reporter
rust/crates/cpd-reporter/src/console_full.rs
+2
View File
@@ -238,6 +238,7 @@ mod tests {
stats: &one_clone_stats(),
duration: Duration::ZERO,
summary: None,
history: None,
};
let result = reporter.report(&[make_clone_no_blame()], &ctx, &PathBuf::from("/tmp"));
assert!(result.is_ok());
@@ -251,6 +252,7 @@ mod tests {
stats: &one_clone_stats(),
duration: Duration::ZERO,
summary: None,
history: None,
};
let result = reporter.report(&[make_clone_with_blame()], &ctx, &PathBuf::from("/tmp"));
assert!(result.is_ok());
rust/crates/cpd-reporter/src/context.rs
+10
View File
@@ -1,3 +1,4 @@
use cpd_core::history::History;
use cpd_core::models::Statistics;
use cpd_core::summary::Summary;
use std::time::Duration;
@@ -14,6 +15,8 @@ pub struct ReportContext<'a> {
pub duration: Duration,
/// Opt-in codebase summary (`--summary`); None when disabled.
pub summary: Option<&'a Summary>,
/// Opt-in duplication trend over git history (`--history`); None when disabled.
pub history: Option<&'a History>,
}
impl<'a> ReportContext<'a> {
@@ -23,6 +26,7 @@ impl<'a> ReportContext<'a> {
stats,
duration,
summary: None,
history: None,
}
}
@@ -31,6 +35,12 @@ impl<'a> ReportContext<'a> {
self.summary = summary;
self
}
/// Attach an opt-in history series.
pub fn with_history(mut self, history: Option<&'a History>) -> Self {
self.history = history;
self
}
}
#[cfg(test)]
rust/crates/cpd-reporter/src/history_render.rs
+220
View File
@@ -0,0 +1,220 @@
// history_render.rs — console rendering for the opt-in `--history` block.
//
// A sparkline of the duplication percentage across the series, a table with
// one row per commit (plus the working tree), the change from the previous
// point colored by direction, and two highlights: the overall trend, and the
// room left under `--threshold`. That last line is what an automatic ratchet
// would apply; jscpd reports it and leaves the decision to the reader.
use crate::shared::Style;
use cpd_core::history::History;
const SUBJECT_WIDTH: usize = 48;
fn truncate(text: &str, width: usize) -> String {
let mut chars = text.chars();
let head: String = chars.by_ref().take(width).collect();
if chars.next().is_some() {
let mut shortened: String = head.chars().take(width.saturating_sub(1)).collect();
shortened.push('…');
shortened
} else {
head
}
}
fn format_change(change: Option<f64>) -> String {
match change {
None => String::new(),
Some(c) if c.abs() < 0.05 => "=".to_string(),
Some(c) => format!("{c:+.1}"),
}
}
fn color_change(text: &str, change: Option<f64>, style: &Style) -> String {
match change {
Some(c) if c >= 0.05 => style.red(text),
Some(c) if c <= -0.05 => style.green_prefix(text),
Some(_) => style.dim(text),
None => text.to_string(),
}
}
/// Full console rendering, appended after the normal reporter output.
pub fn print_history(history: &History, style: &Style) {
println!();
println!(
"{} {}",
style.bold("History"),
style.dim(&format!(
"({}: {} commits + working tree)",
history.range,
history.commit_count()
))
);
if history.points.is_empty() {
return;
}
let pct = history.percentages();
let min = pct.iter().cloned().fold(f64::INFINITY, f64::min);
let max = pct.iter().cloned().fold(f64::NEG_INFINITY, f64::max);
let now = pct[pct.len() - 1];
println!(
" {} {}",
history.sparkline(),
style.dim(&format!("min {min:.1}% max {max:.1}% now {now:.1}%"))
);
let headers = [
"COMMIT",
"DATE",
"FILES",
"LINES",
"CLONES",
"DUP LINES",
"DUP%",
"CHANGE",
"SUBJECT",
];
let rows: Vec<[String; 9]> = history
.points
.iter()
.enumerate()
.map(|(i, p)| {
[
p.short.clone(),
p.date.clone(),
p.sources.to_string(),
p.lines.to_string(),
p.clones.to_string(),
p.duplicated_lines.to_string(),
format!("{:.1}%", p.percentage),
format_change(history.change_at(i)),
if p.is_working_tree() {
"(uncommitted changes)".to_string()
} else {
truncate(&p.subject, SUBJECT_WIDTH)
},
]
})
.collect();
// Left-align the first two and the last column, right-align the numbers.
let mut widths: [usize; 9] = headers.map(str::len);
for row in &rows {
for (w, cell) in widths.iter_mut().zip(row.iter()) {
*w = (*w).max(cell.chars().count());
}
}
let align = |i: usize, cell: &str| -> String {
let width = widths[i];
match i {
0 | 1 => format!("{cell:<width$}"),
8 => cell.to_string(),
_ => format!("{cell:>width$}"),
}
};
let header_line = headers
.iter()
.enumerate()
.map(|(i, h)| align(i, h))
.collect::<Vec<_>>()
.join(" ");
println!(" {}", style.dim(header_line.trim_end()));
for (i, row) in rows.iter().enumerate() {
let change = history.change_at(i);
let line = row
.iter()
.enumerate()
.map(|(col, cell)| {
let text = align(col, cell);
if col == 7 {
color_change(&text, change, style)
} else if col == 8 && history.points[i].is_working_tree() {
style.dim(&text)
} else {
text
}
})
.collect::<Vec<_>>()
.join(" ");
println!(" {}", line.trim_end());
}
if let Some(change) = history.overall_change() {
let first = &history.points[0];
let text = format!(
"Trend: {} points since {} ({})",
format_change(Some(change)),
first.short,
first.date
);
println!("{}", color_change(&text, Some(change), style));
}
if let (Some(threshold), Some(headroom)) = (history.threshold, history.threshold_headroom()) {
println!(
"{}",
style.bold_green(&format!(
"Threshold {threshold:.1}% has {headroom:.1} points of headroom: the series never needed it, tighten it with --threshold {now:.1}"
))
);
}
}
/// Compact rendering for the `ai` reporter: one line per point.
pub fn print_history_compact(history: &History) {
println!(
"history {} (commit/date/files/clones/dup%): {}",
history.range,
history.sparkline()
);
for (i, p) in history.points.iter().enumerate() {
println!(
"{} {} {}/{}/{:.1}{}",
p.short,
p.date,
p.sources,
p.clones,
p.percentage,
match history.change_at(i) {
Some(c) if c.abs() >= 0.05 => format!(" {c:+.1}"),
_ => String::new(),
}
);
}
if let (Some(threshold), Some(headroom)) = (history.threshold, history.threshold_headroom()) {
println!("threshold {threshold:.1} headroom {headroom:.1}");
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn truncate_adds_ellipsis_only_when_cut() {
assert_eq!(truncate("short", 10), "short");
assert_eq!(truncate("exactly-ten", 11), "exactly-ten");
assert_eq!(truncate("a rather long subject line", 10), "a rather …");
}
#[test]
fn change_formatting() {
assert_eq!(format_change(None), "");
assert_eq!(format_change(Some(0.0)), "=");
assert_eq!(format_change(Some(0.04)), "=");
assert_eq!(format_change(Some(1.26)), "+1.3");
assert_eq!(format_change(Some(-0.5)), "-0.5");
}
#[test]
fn change_colors_follow_direction() {
let style = Style::new(false);
assert!(color_change("+1.0", Some(1.0), &style).contains("\x1b[31m"));
assert!(color_change("-1.0", Some(-1.0), &style).contains("\x1b[32m"));
assert!(color_change("=", Some(0.0), &style).contains("\x1b[90m"));
assert_eq!(color_change("x", None, &style), "x");
assert_eq!(color_change("+1.0", Some(1.0), &Style::new(true)), "+1.0");
}
}
rust/crates/cpd-reporter/src/html.rs
+1
View File
@@ -171,6 +171,7 @@ mod tests {
stats,
duration: Duration::ZERO,
summary: None,
history: None,
};
reporter.report(clones, &ctx, &dir).unwrap();
std::fs::read_to_string(dir.join("jscpd-report.html")).unwrap()
rust/crates/cpd-reporter/src/json_reporter.rs
+6
View File
@@ -123,6 +123,11 @@ impl Reporter for JsonReporter {
value["summary"] =
serde_json::to_value(summary).map_err(|e| ReporterError::Format(e.to_string()))?;
}
// Same rule for --history: absent unless requested.
if let Some(history) = ctx.history {
value["history"] =
serde_json::to_value(history).map_err(|e| ReporterError::Format(e.to_string()))?;
}
let content = serde_json::to_string_pretty(&value)
.map_err(|e| ReporterError::Format(e.to_string()))?;
@@ -228,6 +233,7 @@ mod tests {
stats,
duration: Duration::ZERO,
summary: None,
history: None,
};
reporter.report(clones, &ctx, &dir).unwrap();
std::fs::read_to_string(dir.join("jscpd-report.json")).unwrap()
rust/crates/cpd-reporter/src/lib.rs
+1
View File
@@ -6,6 +6,7 @@ pub mod console;
pub mod console_full;
pub mod context;
pub mod csv_reporter;
pub mod history_render;
pub mod html;
pub mod json_reporter;
pub mod markdown_reporter;
rust/crates/cpd-reporter/src/sarif.rs
+1
View File
@@ -559,6 +559,7 @@ mod tests {
stats: &stats,
duration: Duration::ZERO,
summary: None,
history: None,
};
reporter.report(clones, &ctx, &dir).unwrap();
let content = std::fs::read_to_string(dir.join("jscpd-report.sarif")).unwrap();
rust/crates/cpd-reporter/src/silent.rs
+2
View File
@@ -74,6 +74,7 @@ mod tests {
stats: &any_stats(),
duration: Duration::ZERO,
summary: None,
history: None,
};
let result = reporter.report(&[], &ctx, &PathBuf::from("/tmp"));
assert!(result.is_ok());
@@ -103,6 +104,7 @@ mod tests {
stats: &stats,
duration: Duration::ZERO,
summary: None,
history: None,
};
let result = reporter.report(&[], &ctx, &PathBuf::from("/tmp"));
assert!(result.is_ok());
rust/crates/cpd-reporter/src/threshold.rs
+3
View File
@@ -56,6 +56,7 @@ mod tests {
stats: &stats_with_pct(pct, pct as u64),
duration: Duration::ZERO,
summary: None,
history: None,
};
reporter.report(&[], &ctx, &PathBuf::from("/tmp"))
}
@@ -97,6 +98,7 @@ mod tests {
stats: &stats_with_pct(99.9, 99),
duration: Duration::ZERO,
summary: None,
history: None,
};
let result = reporter.report(&[], &ctx, &PathBuf::from("/tmp"));
assert!(result.is_ok(), "no threshold must always return Ok");
@@ -111,6 +113,7 @@ mod tests {
stats: &stats_with_pct(100.0, 100),
duration: Duration::ZERO,
summary: None,
history: None,
};
let result = reporter.report(&[], &ctx, &PathBuf::from("/tmp"));
assert!(result.is_ok(), "silent reporter must always return Ok");
rust/crates/cpd-reporter/tests/blame_integration.rs
+1
View File
@@ -118,6 +118,7 @@ fn run_blame_reporter(
stats: &make_stats(),
duration: Duration::ZERO,
summary: None,
history: None,
};
reporter.report(&[clone], &ctx, &dir).unwrap();
(dir, reporter)
rust/crates/cpd/src/baseline_ref.rs
+24 -13
View File
@@ -40,7 +40,7 @@ pub fn baseline_from_ref(git_ref: &str, run_config: &RunConfig) -> Result<Baseli
result
}
fn git(repo_root: &Path) -> Command {
pub(crate) fn git(repo_root: &Path) -> Command {
let mut cmd = Command::new("git");
cmd.arg("-C").arg(repo_root);
cmd
@@ -64,7 +64,7 @@ fn verify_ref(repo_root: &Path, git_ref: &str) -> Result<(), String> {
Ok(())
}
fn temp_worktree_path() -> PathBuf {
pub(crate) fn temp_worktree_path() -> PathBuf {
// A process-wide counter keeps concurrent runs (e.g. parallel tests) from
// colliding on the same worktree directory.
static NEXT_ID: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
@@ -75,7 +75,7 @@ fn temp_worktree_path() -> PathBuf {
))
}
fn add_worktree(repo_root: &Path, git_ref: &str, worktree: &Path) -> Result<(), String> {
pub(crate) fn add_worktree(repo_root: &Path, git_ref: &str, worktree: &Path) -> Result<(), String> {
let output = git(repo_root)
.args(["worktree", "add", "--detach"])
.arg(worktree)
@@ -94,7 +94,7 @@ fn add_worktree(repo_root: &Path, git_ref: &str, worktree: &Path) -> Result<(),
/// Best-effort cleanup: `git worktree remove` unregisters and deletes in one
/// step; fall back to deleting the directory and pruning the registration.
fn remove_worktree(repo_root: &Path, worktree: &Path) {
pub(crate) fn remove_worktree(repo_root: &Path, worktree: &Path) {
let removed = git(repo_root)
.args(["worktree", "remove", "--force"])
.arg(worktree)
@@ -107,30 +107,41 @@ fn remove_worktree(repo_root: &Path, worktree: &Path) {
}
}
/// Run detection over the base tree with the current run's configuration and
/// fingerprint the clones it contains. Scan paths are remapped from the
/// working tree into the worktree; paths that don't exist in the base ref are
/// simply new code with nothing to record.
fn scan_base_tree(
/// Remap the run's scan paths from the working tree into `worktree`. Paths
/// that do not exist at that ref are skipped: they are new code with nothing
/// to record. `flag` names the option in error messages.
pub(crate) fn map_scan_paths(
run_config: &RunConfig,
repo_root: &Path,
worktree: &Path,
) -> Result<BaselineFile, String> {
let mut base_paths = Vec::new();
flag: &str,
) -> Result<Vec<PathBuf>, String> {
let mut mapped_paths = Vec::new();
for path in &run_config.paths {
let canonical = std::fs::canonicalize(path).unwrap_or_else(|_| path.clone());
let rel = canonical.strip_prefix(repo_root).map_err(|_| {
format!(
"--baseline-from-ref: scan path {} is outside the git repository {}",
"{flag}: scan path {} is outside the git repository {}",
canonical.display(),
repo_root.display()
)
})?;
let mapped = worktree.join(rel);
if mapped.exists() {
base_paths.push(mapped);
mapped_paths.push(mapped);
}
}
Ok(mapped_paths)
}
/// Run detection over the base tree with the current run's configuration and
/// fingerprint the clones it contains.
fn scan_base_tree(
run_config: &RunConfig,
repo_root: &Path,
worktree: &Path,
) -> Result<BaselineFile, String> {
let base_paths = map_scan_paths(run_config, repo_root, worktree, "--baseline-from-ref")?;
if base_paths.is_empty() {
return Ok(BaselineFile::empty());
}
rust/crates/cpd/src/cli.rs
+60
View File
@@ -400,6 +400,24 @@ pub struct Cli {
#[arg(long, value_name = "METRIC")]
pub summary_by: Option<String>,
/// Duplication trend over git history: scan every commit in RANGE (e.g.
/// v5.0.0..HEAD) with this configuration and print a chart and a table
#[arg(long, value_name = "RANGE")]
pub history: Option<String>,
/// Like --history, selecting commits since DATE (e.g. 2026-01-01);
/// combines with --history to bound the range
#[arg(long, value_name = "DATE")]
pub history_since: Option<String>,
/// Keep every Nth commit of the history series, counted from the newest (default: 1)
#[arg(long, value_name = "N")]
pub history_every: Option<usize>,
/// Maximum number of commits in the history series, sampled evenly (default: 30)
#[arg(long, value_name = "N")]
pub history_limit: Option<usize>,
/// Do not write detection progress and result to console
#[arg(long, short = 's')]
pub silent: bool,
@@ -486,6 +504,13 @@ pub struct ConfigFile {
pub summary_top: Option<usize>,
#[serde(alias = "summary-by")]
pub summary_by: Option<String>,
pub history: Option<String>,
#[serde(alias = "history-since")]
pub history_since: Option<String>,
#[serde(alias = "history-every")]
pub history_every: Option<usize>,
#[serde(alias = "history-limit")]
pub history_limit: Option<usize>,
}
#[derive(Debug, Clone, PartialEq)]
@@ -702,6 +727,13 @@ pub(crate) static KNOWN_CONFIG_FIELDS: &[&str] = &[
"sarif-error-tokens",
"fail-on-new-clones",
"baseline-from-ref",
"history",
"historySince",
"historyEvery",
"historyLimit",
"history-since",
"history-every",
"history-limit",
];
pub(crate) static V4_SILENT_IGNORE: &[&str] = &[
@@ -2639,6 +2671,34 @@ mod tests {
assert!(!cli.fail_on_empty);
}
#[test]
fn history_cli_flags() {
let cli = Cli::parse_from([
"cpd",
"--history",
"v5.0.0..HEAD",
"--history-every",
"3",
"--history-limit",
"10",
".",
]);
assert_eq!(cli.history.as_deref(), Some("v5.0.0..HEAD"));
assert_eq!(cli.history_every, Some(3));
assert_eq!(cli.history_limit, Some(10));
let cli = Cli::parse_from(["cpd", "--history-since", "2026-01-01", "."]);
assert_eq!(cli.history_since.as_deref(), Some("2026-01-01"));
assert!(cli.history.is_none());
}
#[test]
fn scan_known_fields_history_keys_are_known() {
assert_no_unknown_diagnostics(
serde_json::json!({"history": "v5..HEAD", "historySince": "2026-01-01", "historyEvery": 2, "historyLimit": 5}),
"history",
);
}
// debug flag
#[test]
fn debug_flag_defaults_to_false() {
rust/crates/cpd/src/history.rs
+262
View File
@@ -0,0 +1,262 @@
// history.rs — duplication trend over git history (`--history`, issue #1002).
//
// For every selected commit, materialize its tree in a temporary detached
// worktree (the same machinery as `--baseline-from-ref`), run detection with
// the current configuration, and keep the totals. Commits come from
// `git log` over a revision range or a `--since` date, oldest first, thinned
// by `--history-every` and capped by `--history-limit` so a long range stays
// a readable series. Scans run one after another; each scan is already
// parallel across files, and parallel worktrees would only compete for the
// same cores.
use crate::baseline_ref::{add_worktree, git, map_scan_paths, remove_worktree};
use crate::options::Options;
use cpd_core::history::{HistoryPoint, WORKING_TREE};
use cpd_core::models::Statistics;
use cpd_finder::orchestrate::{RunConfig, run};
use std::path::Path;
/// What `--history` was asked to walk.
#[derive(Debug, Clone, PartialEq)]
pub struct HistorySpec {
/// `git log` revision range, e.g. `v5.0.0..HEAD`. `HEAD` when only a date
/// was given.
pub range: String,
/// `--since` date passed to `git log`, if any.
pub since: Option<String>,
/// Keep every Nth commit, counted back from the newest.
pub every: usize,
/// Maximum number of commits in the series.
pub limit: usize,
}
impl HistorySpec {
/// `None` when `--history` / `--history-since` were not given.
pub fn from_options(opts: &Options) -> Option<Self> {
if opts.history.is_none() && opts.history_since.is_none() {
return None;
}
Some(Self {
range: opts.history.clone().unwrap_or_else(|| "HEAD".to_string()),
since: opts.history_since.clone(),
every: opts.history_every.max(1),
limit: opts.history_limit.max(1),
})
}
/// Display label for the report header.
pub fn label(&self) -> String {
match &self.since {
Some(since) if self.range == "HEAD" => format!("since {since}"),
Some(since) => format!("{} since {since}", self.range),
None => self.range.clone(),
}
}
}
/// A commit selected for scanning.
#[derive(Debug, Clone, PartialEq)]
pub struct Commit {
pub hash: String,
pub date: String,
pub subject: String,
}
/// List the commits `git log` yields for the spec, oldest first.
pub fn list_commits(repo_root: &Path, spec: &HistorySpec) -> Result<Vec<Commit>, String> {
let mut cmd = git(repo_root);
cmd.args(["log", "--reverse", "--format=%H%x1f%cs%x1f%s"]);
if let Some(since) = &spec.since {
cmd.arg(format!("--since={since}"));
}
cmd.arg(&spec.range).arg("--");
let output = cmd
.output()
.map_err(|e| format!("--history: failed to run git: {e}"))?;
if !output.status.success() {
return Err(format!(
"--history: git log {} failed: {}",
spec.range,
String::from_utf8_lossy(&output.stderr).trim()
));
}
let commits: Vec<Commit> = String::from_utf8_lossy(&output.stdout)
.lines()
.filter_map(|line| {
let mut parts = line.splitn(3, '\u{1f}');
let hash = parts.next()?.trim();
if hash.is_empty() {
return None;
}
Some(Commit {
hash: hash.to_string(),
date: parts.next().unwrap_or("").to_string(),
subject: parts.next().unwrap_or("").to_string(),
})
})
.collect();
if commits.is_empty() {
return Err(format!(
"--history: no commits match {} in {}",
spec.label(),
repo_root.display()
));
}
Ok(commits)
}
/// Thin a series: keep every Nth item counted back from the newest (so the
/// newest always stays), then cap the length by evenly spaced sampling that
/// keeps both ends.
pub fn sample<T: Clone>(items: &[T], every: usize, limit: usize) -> Vec<T> {
let every = every.max(1);
let n = items.len();
let thinned: Vec<T> = items
.iter()
.enumerate()
.filter(|(i, _)| (n - 1 - i).is_multiple_of(every))
.map(|(_, item)| item.clone())
.collect();
let limit = limit.max(1);
let m = thinned.len();
if m <= limit {
return thinned;
}
if limit == 1 {
return vec![thinned[m - 1].clone()];
}
(0..limit)
.map(|k| thinned[k * (m - 1) / (limit - 1)].clone())
.collect()
}
/// Scan every selected commit and return one point per commit, oldest first.
pub fn collect_history(
spec: &HistorySpec,
run_config: &RunConfig,
) -> Result<Vec<HistoryPoint>, String> {
let first = run_config
.paths
.first()
.ok_or("--history: no scan paths given")?;
let repo_root = crate::find_git_root(first).ok_or_else(|| {
format!(
"--history: {} is not inside a git repository",
first.display()
)
})?;
let commits = sample(&list_commits(&repo_root, spec)?, spec.every, spec.limit);
let mut points = Vec::with_capacity(commits.len());
for commit in &commits {
let worktree = crate::baseline_ref::temp_worktree_path();
add_worktree(&repo_root, &commit.hash, &worktree)
.map_err(|e| e.replace("--baseline-from-ref", "--history"))?;
let result = scan_commit(run_config, &repo_root, &worktree);
remove_worktree(&repo_root, &worktree);
let stats = result?;
points.push(point_from_stats(
commit.hash.clone(),
commit.hash.chars().take(7).collect(),
commit.date.clone(),
commit.subject.clone(),
&stats,
));
}
Ok(points)
}
/// The series' last point: the current run's own totals.
pub fn working_tree_point(stats: &Statistics) -> HistoryPoint {
point_from_stats(
WORKING_TREE.to_string(),
"working".to_string(),
stats.detection_date.chars().take(10).collect(),
String::new(),
stats,
)
}
fn point_from_stats(
commit: String,
short: String,
date: String,
subject: String,
stats: &Statistics,
) -> HistoryPoint {
let t = &stats.total;
HistoryPoint {
commit,
short,
date,
subject,
sources: t.sources,
lines: t.lines,
tokens: t.tokens,
clones: t.clones,
duplicated_lines: t.duplicated_lines,
percentage: t.percentage,
}
}
fn scan_commit(
run_config: &RunConfig,
repo_root: &Path,
worktree: &Path,
) -> Result<Statistics, String> {
let paths = map_scan_paths(run_config, repo_root, worktree, "--history")?;
if paths.is_empty() {
// The scan paths did not exist at this commit: an honest zero.
return Ok(Statistics {
total: Default::default(),
formats: Default::default(),
detection_date: String::new(),
});
}
let config = RunConfig {
paths,
blame: false,
..run_config.clone()
};
run(&config)
.map(|r| r.statistics)
.map_err(|e| format!("--history: scan failed: {e}"))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn sample_keeps_newest_when_thinning() {
let items: Vec<u32> = (1..=10).collect();
assert_eq!(sample(&items, 3, 100), vec![1, 4, 7, 10]);
assert_eq!(sample(&items, 1, 100), items);
}
#[test]
fn sample_caps_length_keeping_both_ends() {
let items: Vec<u32> = (1..=10).collect();
assert_eq!(sample(&items, 1, 4), vec![1, 4, 7, 10]);
assert_eq!(sample(&items, 1, 1), vec![10]);
assert_eq!(sample(&items, 1, 2), vec![1, 10]);
assert_eq!(sample(&[1u32], 5, 5), vec![1]);
assert!(sample(&Vec::<u32>::new(), 1, 3).is_empty());
}
#[test]
fn label_describes_range_and_since() {
let spec = |range: &str, since: Option<&str>| HistorySpec {
range: range.to_string(),
since: since.map(str::to_string),
every: 1,
limit: 30,
};
assert_eq!(spec("v5.0.0..HEAD", None).label(), "v5.0.0..HEAD");
assert_eq!(spec("HEAD", Some("2026-01-01")).label(), "since 2026-01-01");
assert_eq!(
spec("main", Some("2026-01-01")).label(),
"main since 2026-01-01"
);
}
}
rust/crates/cpd/src/main.rs
+33 -1
View File
@@ -1,5 +1,6 @@
mod baseline_ref;
mod cli;
mod history;
mod mcp;
mod options;
mod timer;
@@ -69,6 +70,10 @@ struct MergedConfig {
summary: bool,
summary_top: usize,
summary_by: String,
history: Option<String>,
history_since: Option<String>,
history_every: usize,
history_limit: usize,
}
impl MergedConfig {
@@ -122,6 +127,10 @@ impl MergedConfig {
summary: opts.summary,
summary_top: opts.summary_top,
summary_by: opts.summary_by.to_string(),
history: opts.history.clone(),
history_since: opts.history_since.clone(),
history_every: opts.history_every,
history_limit: opts.history_limit,
}
}
}
@@ -474,6 +483,27 @@ fn main() {
None
};
// Opt-in duplication trend over git history (#1002): one scan per
// selected commit in a temporary worktree, plus the current run as the
// last point. Nothing here runs without --history / --history-since.
let history = match history::HistorySpec::from_options(&opts) {
None => None,
Some(spec) => match history::collect_history(&spec, &run_config) {
Ok(mut points) => {
points.push(history::working_tree_point(&statistics));
Some(cpd_core::history::History {
range: spec.label(),
threshold: opts.threshold,
points,
})
}
Err(e) => {
eprintln!("Error: {}", e);
std::process::exit(1);
}
},
};
// Reporter options
let reporter_opts = ReporterOptions {
output_dir: opts.output_dir.clone(),
@@ -536,7 +566,9 @@ fn main() {
}
};
let ctx = ReportContext::new(&statistics, elapsed).with_summary(summary.as_ref());
let ctx = ReportContext::new(&statistics, elapsed)
.with_summary(summary.as_ref())
.with_history(history.as_ref());
match reporter.report(&clones, &ctx, &opts.output_dir) {
Ok(()) => {}
Err(cpd_reporter::reporter::ReporterError::ThresholdExceeded {
rust/crates/cpd/src/options.rs
+8
View File
@@ -51,6 +51,10 @@ pub struct Options {
pub summary: bool,
pub summary_top: usize,
pub summary_by: SummaryMetric,
pub history: Option<String>,
pub history_since: Option<String>,
pub history_every: usize,
pub history_limit: usize,
pub pattern: Option<String>,
#[allow(dead_code)]
pub list: bool,
@@ -194,6 +198,10 @@ impl Options {
.any(|var| std::env::var(var).is_ok()),
silent: cli.silent || config.silent.unwrap_or(false),
summary: cli.summary || config.summary.unwrap_or(false),
history: cli.history.clone().or(config.history.clone()),
history_since: cli.history_since.clone().or(config.history_since.clone()),
history_every: cli.history_every.or(config.history_every).unwrap_or(1),
history_limit: cli.history_limit.or(config.history_limit).unwrap_or(30),
summary_top: cli.summary_top.or(config.summary_top).unwrap_or(10),
// Invalid metric values are warned about in main() (like --mode).
summary_by: cli
rust/crates/cpd/tests/integration.rs
+231
View File
@@ -2142,3 +2142,234 @@ fn reporter_write_failure_exits_one() {
stderr
);
}
// ── --history: duplication trend over git history (#1002) ─────────────────
/// Repo with four commits: no clone, one clone (b.js copies a.js), two
/// clones (c.js copies too), one clone again (b.js rewritten). Dates are
/// fixed so the series is deterministic.
fn setup_history_repo() -> PathBuf {
let root = baseline_tmp_dir("history");
let src = root.join("src");
std::fs::create_dir_all(&src).unwrap();
assert!(git_in(&root, &["init", "-q"]).status.success());
let commit = |msg: &str, date: &str| {
assert!(git_in(&root, &["add", "-A"]).status.success());
let out = Command::new("git")
.arg("-C")
.arg(&root)
.args([
"-c",
"user.email=cpd-test@example.com",
"-c",
"user.name=cpd-test",
"-c",
"commit.gpgsign=false",
"commit",
"-q",
"-m",
msg,
"--date",
date,
])
.env("GIT_COMMITTER_DATE", date)
.output()
.expect("failed to run git");
assert!(
out.status.success(),
"git commit failed: {}",
String::from_utf8_lossy(&out.stderr)
);
};
std::fs::write(src.join("a.js"), dup_function("total")).unwrap();
commit("initial", "2026-08-01T10:00:00");
std::fs::write(src.join("b.js"), dup_function("total")).unwrap();
commit("copy into b", "2026-08-08T10:00:00");
std::fs::write(src.join("c.js"), dup_function("total")).unwrap();
commit("copy into c", "2026-08-15T10:00:00");
std::fs::write(src.join("b.js"), "export const rate = (n) => n * 7;\n").unwrap();
commit("rewrite b", "2026-08-22T10:00:00");
root
}
fn run_history_cpd(root: &std::path::Path, extra: &[&str]) -> Output {
let scan = root.join("src");
let mut args = vec!["--min-tokens", "20", "--no-colors", "--no-tips"];
args.extend_from_slice(extra);
args.push(scan.to_str().unwrap());
run_cpd(args).expect("cpd binary must exist")
}
#[test]
fn history_json_has_one_point_per_commit_plus_working_tree() {
let Some(_) = maybe_bin() else { return };
let root = setup_history_repo();
let out_dir = root.join("report");
let output = run_history_cpd(
&root,
&[
"--history-since",
"2026-01-01",
"--reporters",
"json",
"--output",
out_dir.to_str().unwrap(),
],
);
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(output.status.success(), "stderr: {stderr}");
let report: serde_json::Value =
serde_json::from_str(&std::fs::read_to_string(out_dir.join("jscpd-report.json")).unwrap())
.unwrap();
std::fs::remove_dir_all(&root).ok();
let history = &report["history"];
assert_eq!(history["range"], "since 2026-01-01");
let points = history["points"].as_array().unwrap();
assert_eq!(points.len(), 5, "4 commits + working tree: {points:?}");
let clones: Vec<u64> = points
.iter()
.map(|p| p["clones"].as_u64().unwrap())
.collect();
assert_eq!(clones, vec![0, 1, 2, 1, 1]);
let dates: Vec<&str> = points.iter().map(|p| p["date"].as_str().unwrap()).collect();
assert_eq!(
&dates[..4],
&["2026-08-01", "2026-08-08", "2026-08-15", "2026-08-22"]
);
assert_eq!(points[1]["subject"], "copy into b");
assert_eq!(points[4]["commit"], "working tree");
assert!(points[2]["percentage"].as_f64().unwrap() > points[1]["percentage"].as_f64().unwrap());
}
#[test]
fn history_console_prints_chart_table_and_threshold_hint() {
let Some(_) = maybe_bin() else { return };
let root = setup_history_repo();
let output = run_history_cpd(&root, &["--history", "HEAD~3..HEAD", "--threshold", "80"]);
std::fs::remove_dir_all(&root).ok();
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(
output.status.success(),
"stderr: {}",
String::from_utf8_lossy(&output.stderr)
);
assert!(
stdout.contains("History (HEAD~3..HEAD: 3 commits + working tree)"),
"{stdout}"
);
assert!(
stdout.contains("min ") && stdout.contains("now "),
"{stdout}"
);
assert!(
stdout.chars().any(|c| "▁▂▃▄▅▆▇█".contains(c)),
"sparkline missing: {stdout}"
);
assert!(stdout.contains("CHANGE SUBJECT"), "{stdout}");
assert!(stdout.contains("(uncommitted changes)"), "{stdout}");
assert!(stdout.contains("Trend: "), "{stdout}");
assert!(stdout.contains("tighten it with --threshold"), "{stdout}");
}
#[test]
fn history_every_and_limit_thin_the_series() {
let Some(_) = maybe_bin() else { return };
let root = setup_history_repo();
let out_dir = root.join("report");
let output = run_history_cpd(
&root,
&[
"--history-since",
"2026-01-01",
"--history-every",
"2",
"--reporters",
"json",
"--output",
out_dir.to_str().unwrap(),
],
);
assert!(output.status.success());
let report: serde_json::Value =
serde_json::from_str(&std::fs::read_to_string(out_dir.join("jscpd-report.json")).unwrap())
.unwrap();
let dates: Vec<String> = report["history"]["points"]
.as_array()
.unwrap()
.iter()
.filter(|p| p["commit"] != "working tree")
.map(|p| p["date"].as_str().unwrap().to_string())
.collect();
assert_eq!(
dates,
vec!["2026-08-08", "2026-08-22"],
"every 2nd, newest kept"
);
let output = run_history_cpd(
&root,
&[
"--history-since",
"2026-01-01",
"--history-limit",
"2",
"--reporters",
"json",
"--output",
out_dir.to_str().unwrap(),
],
);
assert!(output.status.success());
let report: serde_json::Value =
serde_json::from_str(&std::fs::read_to_string(out_dir.join("jscpd-report.json")).unwrap())
.unwrap();
std::fs::remove_dir_all(&root).ok();
let dates: Vec<String> = report["history"]["points"]
.as_array()
.unwrap()
.iter()
.filter(|p| p["commit"] != "working tree")
.map(|p| p["date"].as_str().unwrap().to_string())
.collect();
assert_eq!(
dates,
vec!["2026-08-01", "2026-08-22"],
"limit keeps both ends"
);
}
#[test]
fn history_without_matching_commits_is_an_error() {
let Some(_) = maybe_bin() else { return };
let root = setup_history_repo();
let output = run_history_cpd(&root, &["--history", "nosuchref..HEAD"]);
std::fs::remove_dir_all(&root).ok();
let stderr = String::from_utf8_lossy(&output.stderr);
assert_eq!(output.status.code(), Some(1), "stderr: {stderr}");
assert!(
stderr.contains("--history: git log nosuchref..HEAD failed"),
"{stderr}"
);
}
#[test]
fn history_outside_a_git_repository_is_an_error() {
let Some(_) = maybe_bin() else { return };
let dir = scratch_dir("history-nogit");
std::fs::write(dir.join("a.js"), dup_function("x")).unwrap();
let output = run_cpd([
dir.as_os_str(),
std::ffi::OsStr::new("--history"),
std::ffi::OsStr::new("HEAD"),
std::ffi::OsStr::new("--reporters"),
std::ffi::OsStr::new("silent"),
])
.expect("cpd binary must exist");
std::fs::remove_dir_all(&dir).ok();
let stderr = String::from_utf8_lossy(&output.stderr);
assert_eq!(output.status.code(), Some(1), "stderr: {stderr}");
assert!(
stderr.contains("is not inside a git repository"),
"{stderr}"
);
}