fix(test-cache): close three false-skip paths found by review

Adversarial review of af72b46854 confirmed three ways a failing test could be skipped green, each reproduced on a fixture: - An input edited while the run was in flight was fingerprinted at the end and recorded as the version that passed. Records are now refused for any input modified after the run started (directories by their subtree). - A run that failed on an unhandled error still recorded its files, because the module state stays "passed". Nothing is recorded from a run with unhandled errors, or an interrupted one. - A computed import in a HELPER (`import(/* @vite-ignore */ file)`, the form pending-review-mute.test.ts uses) was invisible to the key. Every first-party module in the closure is now scanned, the comment form is matched, and importing child_process/worker_threads/cluster in the closure keeps a file uncached however it is called. Also: a false skip now deletes that file's records, so clearing TRIPPED cannot revive it; TRIPPED is written first and atomically, and an unreadable marker reads as tripped; package.json files and pnpm's installed lock are in the salt; a sibling that would shadow an import (foo.ts beside foo/index.ts) changes the key; the sequencer skips nothing on a file-filtered run or when the cache reporter is not loaded; messages go to stderr; a malformed sample rate falls back to 5%; the fs tracker loads first and covers access/open/ realpath/readlink. Two regressions inside this round, caught by a positive control that ordinary tests still record: scanning the fs tracker's own closure (which imports child_process) marked every test uncacheable, and a call-name pattern matched `regex.exec(` in src/__tests__/setup.ts. The tracker is excluded as instrumentation, and process use is detected by import, not call name. Fixture battery, positive control first (recorded 2, then skipped 2): edited mid-run not recorded and next run fails as it should; leaked rejection blocks recording; helper computed import and namespaced child_process not recorded; shadowing file re-runs; false skip trips and forgets. 89-file yardstick: cold 63s, warm 14s, 84 skipped, 0 false skips; the two files kept uncached are pending-review-mute (the reviewer's case) and a computed-import hook. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 22:08:18 +08:00 · 2026-09-19 00:43:12 -06:00
parent af72b46854
commit fbe832cbdc
7 changed files with 403 additions and 141 deletions
@@ -187,8 +187,8 @@ every file it read, and the lockfile/configs. Results are shared between worktre

 A random ~5% of the unchanged files run anyway. If one of those fails, the cache predicted a pass it could not
 deliver — a **false skip** — and it trips itself off (`TRIPPED.json` in the cache dir) until a human looks.
-**Known blind spot:** environment variables are not part of the key. Never on in CI; never applied to a run
-that names files. Code and details: `scripts/test-cache/`.
+**Known blind spot:** environment variables are not part of the key. Never on in CI; a run that filters
+files (a filename, directory or substring) is never trimmed. Code and details: `scripts/test-cache/`.

 #### Worker count: uncapped by default, `VITEST_MAX_WORKERS` / `--max-workers` to size it
 A suite uses Vitest's own worker count (`cpus - 1` in run mode, `floor(cpus / 2)` in watch; the browser pool `min(12, cpus - 1)`).
@@ -714,7 +714,7 @@ about your change, whatever the summary says.
 On a queued run with the result cache on, that file can be absent from the output for a legitimate reason:
 it was skipped as unchanged since it last passed. The `[test-cache]` line names how many files were skipped.
 To apply this check, confirm the file either ran with a nonzero count or is not in your diff's reach —
-or run it by name, which never goes through the cache.
+or run it by its full filename, which never goes through the cache.

 **A fresh worktree also has no `.envrc`.** It's gitignored, so it never comes with the checkout, and you silently
 get system Node instead of the flake's pinned version. Measured (when the flake still shipped node 22): system
@@ -8,25 +8,38 @@ import * as Core from '../test-cache/core.mjs';
 type Node = { id: string; importedModules: Set<Node> };
 type Fingerprint = (rel: string) => string;

-const { closureOf, toRel, isCoveredElsewhere, alwaysRuns, keyFor, makeFingerprinter, mode } =
-  Core as unknown as {
-    closureOf: (
-      g: { getModuleById: (id: string) => Node | undefined },
-      id: string
-    ) => Set<string> | null;
-    toRel: (id: string, root: string) => string | null;
-    isCoveredElsewhere: (rel: string | null) => boolean;
-    alwaysRuns: (source: string) => boolean;
-    keyFor: (a: {
-      salt: string;
-      project: string;
-      testRel: string;
-      entries: string[];
-      fingerprint: Fingerprint;
-    }) => string;
-    makeFingerprinter: (root: string) => Fingerprint;
-    mode: (env: Record<string, string | undefined>) => string;
-  };
+const {
+  closureOf,
+  toRel,
+  isCoveredElsewhere,
+  alwaysRuns,
+  keyFor,
+  makeFingerprinter,
+  mode,
+  shadowCandidates,
+  changedSince,
+  tripped,
+} = Core as unknown as {
+  closureOf: (
+    g: { getModuleById: (id: string) => Node | undefined },
+    id: string
+  ) => Set<string> | null;
+  toRel: (id: string, root: string) => string | null;
+  isCoveredElsewhere: (rel: string | null) => boolean;
+  alwaysRuns: (source: string) => boolean;
+  keyFor: (a: {
+    salt: string;
+    project: string;
+    testRel: string;
+    entries: string[];
+    fingerprint: Fingerprint;
+  }) => string;
+  makeFingerprinter: (root: string) => Fingerprint;
+  mode: (env: Record<string, string | undefined>) => string;
+  shadowCandidates: (rel: string) => string[];
+  changedSince: (root: string, rel: string, sinceMs: number) => boolean;
+  tripped: (dir: string) => { at: string } | null;
+};

 function graphOf(edges: Record<string, string[]>) {
  const nodes = new Map<string, Node>();
@@ -107,9 +120,13 @@ describe('the key', () => {

 describe('tests that always run', () => {
  it.each([
-    ["spawn(process.execPath, ['x.mjs']);"],
+    ["import { execSync } from 'node:child_process';"],
+    ["import * as cp from 'child_process';"],
+    ["const { Worker } = require('worker_threads');"],
    ['const m = await import(`./pages/${name}`);'],
    ['const m = await import(target);'],
+    // The form this repo uses, which the first version of the pattern let through.
+    ['return import(/* @vite-ignore */ file);'],
    ["const files = globSync('src/**/*.ts');"],
    ["import fg from 'fast-glob';"],
  ])('recognises %s', (source) => {
@@ -118,12 +135,15 @@ describe('tests that always run', () => {

  // Plain file reads are no longer a reason to always run: the tracker records them into the key.
  // A literal dynamic import is in the module graph.
-  it.each([["readFileSync('src/x.ts', 'utf8');"], ["await import('./dep');"]])(
-    'leaves %s cacheable',
-    (source) => {
-      expect(alwaysRuns(source)).toBe(false);
-    }
-  );
+  // `regex.exec(` is not a process: the call-name pattern this replaced matched it in
+  // src/__tests__/setup.ts, which every test loads, and nothing was cacheable at all.
+  it.each([
+    ["readFileSync('src/x.ts', 'utf8');"],
+    ["await import('./dep');"],
+    ['const m = /^a(b)$/.exec(input);'],
+  ])('leaves %s cacheable', (source) => {
+    expect(alwaysRuns(source)).toBe(false);
+  });
 });

 // CI must always run everything: it is the check that still runs when nothing local does.
@@ -138,3 +158,53 @@ describe('when the cache is active', () => {
    expect(mode({ CIVITAI_TEST_CACHE: 'on' })).toBe('on');
  });
 });
+
+describe('what a key must see besides content', () => {
+  // `./foo` resolving to `foo/index.ts` is overtaken by a new `foo.ts` without any file in the
+  // dependency list changing.
+  it('names the files that would shadow an index module or an extension', () => {
+    const c = shadowCandidates('src/foo/index.ts');
+    expect(c).toContain('src/foo.ts');
+    expect(c).toContain('src/foo/index.tsx');
+    expect(c).not.toContain('src/foo/index.ts');
+  });
+
+  it('changes when a shadowing file appears', () => {
+    const root = mkdtempSync(join(tmpdir(), 'test-cache-shadow-'));
+    mkdirSync(join(root, 'src/foo'), { recursive: true });
+    writeFileSync(join(root, 't.test.ts'), 't');
+    writeFileSync(join(root, 'src/foo/index.ts'), 'i');
+    const k = () =>
+      keyFor({
+        salt: 's',
+        project: 'unit',
+        testRel: 't.test.ts',
+        entries: ['src/foo/index.ts'],
+        fingerprint: makeFingerprinter(root),
+      });
+    const before = k();
+    writeFileSync(join(root, 'src/foo.ts'), 'f');
+    expect(k()).not.toBe(before);
+  });
+
+  // The key is computed at the END of a run; an input written after the run started is not what
+  // ran. Measured by review: without this an edit made during a queued suite was recorded as passing.
+  it('sees a file written after a given instant, and a directory whose subtree gained a file', () => {
+    const root = mkdtempSync(join(tmpdir(), 'test-cache-mtime-'));
+    mkdirSync(join(root, 'd/deep'), { recursive: true });
+    writeFileSync(join(root, 'a.ts'), 'a');
+    const since = Date.now() + 60_000;
+    expect(changedSince(root, 'a.ts', since)).toBe(false);
+    expect(changedSince(root, 'a.ts', 0)).toBe(true);
+    expect(changedSince(root, 'd', since)).toBe(false);
+    expect(changedSince(root, 'missing.ts', 0)).toBe(false);
+  });
+
+  // A half-written marker must read as tripped, never as "safe to skip".
+  it('treats an unreadable trip marker as tripped', () => {
+    const dir = mkdtempSync(join(tmpdir(), 'test-cache-trip-'));
+    expect(tripped(dir)).toBeNull();
+    writeFileSync(join(dir, 'TRIPPED.json'), '{"at": "2026');
+    expect(tripped(dir)).not.toBeNull();
+  });
+});
@@ -35,15 +35,31 @@ const HERE = dirname(fileURLToPath(import.meta.url));

 // Inputs every test depends on that no import edge or file read records. This cache's own code is
 // among them: a fix to how reads are captured must invalidate everything recorded without it.
-const GLOBAL_INPUTS = ['pnpm-lock.yaml', 'vitest.config.mts', 'tsconfig.json'];
+// `node_modules/.pnpm/lock.yaml` is what pnpm actually INSTALLED, which a rebased tree that skipped
+// `pnpm install` does not share with its lockfile. The package.json files carry `exports` maps, and a
+// symlinked workspace package resolves through them — the lockfile does not record `exports`.
+const GLOBAL_INPUTS = [
+  'pnpm-lock.yaml',
+  'node_modules/.pnpm/lock.yaml',
+  'package.json',
+  'vitest.config.mts',
+  'tsconfig.json',
+];
+const WORKSPACE_DIRS = ['packages', 'apps'];
 const OWN_CODE = ['core.mjs', 'fs-tracker.mjs', 'sequencer.mjs', 'reporter.mjs'];

 // What stays uncacheable even with file reads tracked: a child process reads what it likes, and a
 // dynamic import whose specifier is computed is invisible to the module graph. 19 files, 0.7% of
 // modelled worker time, measured 2026-09-19.
 const ALWAYS_RUN_SOURCE = [
-  /\b(?:spawn|spawnSync|execSync|execFileSync|execFile)\(/,
-  /import\(\s*(?:`[^`]*\$\{|[A-Za-z_$])/,
+  // IMPORTING a process or thread module, not calling one by name: a call pattern cannot see
+  // `cp.execSync(` or a destructured alias, and `exec(` matched every `regex.exec(` in the repo —
+  // measured: src/__tests__/setup.ts tripped it and nothing was cacheable at all.
+  /\bfrom\s+['"](?:node:)?(?:child_process|worker_threads|cluster)['"]/,
+  /\b(?:require|import)\s*\(\s*['"](?:node:)?(?:child_process|worker_threads|cluster)['"]\s*\)/,
+  // Comments allowed between `import(` and the specifier: `import(/* @vite-ignore */ file)` is the
+  // form this repo actually uses, and the first version of this pattern let it through.
+  /import\(\s*(?:\/\*[\s\S]*?\*\/\s*)*(?:`[^`]*\$\{|[A-Za-z_$])/,
  // A glob's result changes when a MATCHING file is added anywhere, and a pattern is not a path
  // whose state can be fingerprinted.
  /\b(?:globSync|glob|globby|fastGlob|fg)\(|from ['"](?:fast-glob|globby|glob|tinyglobby)['"]/,
@@ -82,6 +98,40 @@ export function alwaysRuns(source) {
  return ALWAYS_RUN_SOURCE.some((re) => re.test(source));
 }

+// Reaching one of these anywhere in the graph means a process or thread the key cannot see into.
+// Checked on the graph rather than by pattern, so an aliased or destructured call cannot hide it.
+const OPAQUE_BUILTINS = new Set(['child_process', 'worker_threads', 'cluster']);
+
+export function reachesOpaqueBuiltin(ids) {
+  for (const id of ids) {
+    const bare = String(id).replace(/^node:/, '');
+    if (OPAQUE_BUILTINS.has(bare)) return true;
+  }
+  return false;
+}
+
+const RESOLVABLE_EXTS = ['ts', 'tsx', 'mts', 'cts', 'js', 'jsx', 'mjs', 'cjs', 'json'];
+
+/**
+ * Paths whose APPEARANCE would change what an import of `rel` resolves to. `./foo` resolving to
+ * `foo/index.ts` is overtaken by a new `foo.ts`; `foo.ts` is overtaken by an extension ahead of it.
+ * Neither edits a file already in the dependency list, so without these the key cannot see it.
+ * Only these siblings, not the whole directory: a listing would invalidate every test near any
+ * new file.
+ */
+export function shadowCandidates(rel) {
+  const m = /^(.*?)([^/]+)\.([^./]+)$/.exec(rel);
+  if (!m) return [];
+  const [, dir, name, ext] = m;
+  if (!RESOLVABLE_EXTS.includes(ext)) return [];
+  const out = RESOLVABLE_EXTS.filter((e) => e !== ext).map((e) => `${dir}${name}.${e}`);
+  if (name === 'index' && dir) {
+    const parent = dir.replace(/\/$/, '');
+    for (const e of RESOLVABLE_EXTS) out.push(`${parent}.${e}`);
+  }
+  return out;
+}
+
 /** Transitive imports in one vite environment's module graph. null when the root is absent. */
 export function closureOf(graph, id) {
  const root = graph.getModuleById(id);
@@ -140,6 +190,20 @@ function listTree(abs) {
  return out.sort();
 }

+function workspaceManifests(root) {
+  const out = [];
+  for (const ws of WORKSPACE_DIRS) {
+    let names = [];
+    try {
+      names = readdirSync(join(root, ws));
+    } catch {
+      continue;
+    }
+    for (const n of names.sort()) out.push(`${ws}/${n}/package.json`);
+  }
+  return out;
+}
+
 export function globalSalt(root, vitestVersion, fingerprint = makeFingerprinter(root)) {
  const own = OWN_CODE.map((f) => {
    try {
@@ -155,6 +219,7 @@ export function globalSalt(root, vitestVersion, fingerprint = makeFingerprinter(
      process.platform,
      vitestVersion,
      GLOBAL_INPUTS.map((f) => [f, fingerprint(f)]),
+      workspaceManifests(root).map((f) => [f, fingerprint(f)]),
      own,
    ])
  );
@@ -162,11 +227,37 @@ export function globalSalt(root, vitestVersion, fingerprint = makeFingerprinter(

 /** The key. `entries` is every repo-relative path the file depends on; order does not matter. */
 export function keyFor({ salt, project, testRel, entries, fingerprint }) {
-  const parts = [...new Set(entries)]
-    .filter((rel) => !isCoveredElsewhere(rel))
-    .sort()
-    .map((rel) => `${rel}\0${fingerprint(rel)}`);
-  return sha([salt, project, testRel, fingerprint(testRel), ...parts].join('\n'));
+  const deps = [...new Set(entries)].filter((rel) => !isCoveredElsewhere(rel)).sort();
+  const parts = deps.map((rel) => `${rel}\0${fingerprint(rel)}`);
+  // Only the shadow candidates that EXIST, so the common case (none) adds nothing to fingerprint.
+  const shadows = [...new Set(deps.flatMap(shadowCandidates))]
+    .filter((rel) => fingerprint(rel) !== 'missing')
+    .sort();
+  return sha([salt, project, testRel, fingerprint(testRel), ...parts, '--shadows--', ...shadows].join('\n'));
+}
+
+/**
+ * Whether `rel` was modified at or after `sinceMs`. A directory counts as modified when any
+ * directory in its subtree was — that is what moves when a file is added, removed or renamed.
+ * Used to refuse recording a pass whose inputs changed while the run was in flight: the key is
+ * computed from disk at the END of a run, and an agent editing during a queued suite would
+ * otherwise have its edit recorded as the version that passed. Measured by review: exactly that.
+ */
+export function changedSince(root, rel, sinceMs) {
+  const abs = join(root, rel);
+  let st;
+  try {
+    st = statSync(abs);
+  } catch {
+    return false; // absent now; its absence is what the key records
+  }
+  if (st.mtimeMs >= sinceMs || st.ctimeMs >= sinceMs) return true;
+  if (!st.isDirectory()) return false;
+  for (const e of readdirSync(abs, { withFileTypes: true })) {
+    if (!e.isDirectory() || e.name === 'node_modules' || e.name === '.git') continue;
+    if (changedSince(root, `${rel}/${e.name}`, sinceMs)) return true;
+  }
+  return false;
 }

 export const identity = (project, testRel) => sha(`${project}\0${testRel}`);
@@ -220,10 +311,25 @@ export function writeRecord(dir, project, testRel, record) {

 export const trippedPath = (dir) => join(dir, 'TRIPPED.json');

+/** A marker that exists but cannot be parsed still means tripped — the safe reading of a half-write. */
 export function tripped(dir) {
+  const p = trippedPath(dir);
+  if (!existsSync(p)) return null;
  try {
-    return JSON.parse(readFileSync(trippedPath(dir), 'utf8'));
+    return JSON.parse(readFileSync(p, 'utf8'));
  } catch {
-    return null;
+    return { at: 'unknown (marker unreadable)', falseSkips: [] };
  }
 }
+
+export function writeTripped(dir, body) {
+  const p = trippedPath(dir);
+  const tmp = `${p}.${process.pid}.tmp`;
+  writeFileSync(tmp, JSON.stringify(body, null, 2));
+  renameSync(tmp, p);
+}
+
+/** Every record of a file — used on a false skip, so clearing TRIPPED cannot revive the same skip. */
+export function forget(dir, project, testRel) {
+  rmSync(join(dir, 'rec', identity(project, testRel)), { recursive: true, force: true });
+}
@@ -51,10 +51,18 @@ if (mode() !== 'off') {
    'opendirSync',
    'opendir',
    'createReadStream',
+    'accessSync',
+    'access',
+    'openSync',
+    'open',
+    'realpathSync',
+    'realpath',
+    'readlinkSync',
+    'readlink',
  ]) {
    wrap(fs, name);
  }
-  for (const name of ['readFile', 'readdir', 'stat', 'lstat', 'opendir']) {
+  for (const name of ['readFile', 'readdir', 'stat', 'lstat', 'opendir', 'open', 'access', 'realpath', 'readlink']) {
    wrap(fs.promises, name);
  }
  syncBuiltinESMExports();
@@ -1,22 +1,21 @@
 /**
 * Records, AFTER a run, which test files passed and what they depended on — and checks the cache's
- * own work. Pairs with scripts/test-cache/sequencer.mjs, which read these records before the run.
+ * own work. Pairs with scripts/test-cache/sequencer.mjs, which reads these records before the run.
 *
- * Dependencies come from vite's server-side ssr module graph, not `diagnostic().importDurations`:
- * measured on a fixture, importDurations recorded a static import and MISSED an `await import()`
- * made inside a test body, while the ssr graph caught both, cold and warm. File reads come from
+ * Dependencies come from vite's module graph, not `diagnostic().importDurations`: measured on a
+ * fixture, importDurations recorded a static import and MISSED an `await import()` made inside a
+ * test body, while the graph caught both, cold and warm. File reads come from
 * scripts/test-cache/fs-tracker.mjs via the file task's meta.
 *
 * The check: every file the sequencer found unchanged but ran anyway — all of them in `shadow`
 * mode, the random sample in `on` — was predicted to pass. One that fails is a FALSE SKIP: the key
- * missed a dependency, and in `on` mode that file would have been reported green without running.
- * The reporter then trips the cache so every later run executes in full until a human looks.
+ * missed a dependency. The reporter trips the cache, so every later run executes in full until a
+ * human looks, and deletes that file's records, so clearing the trip cannot revive the same skip.
 *
- * 🔴 This file must never change a run's outcome. Everything is caught and logged. It imports node
- * builtins only.
+ * 🔴 This file must never change a run's outcome. Everything is caught and logged, to stderr.
 */

-import { appendFileSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { appendFileSync, mkdirSync, readFileSync } from 'node:fs';
 import { join } from 'node:path';

 import * as core from './core.mjs';
@@ -30,106 +29,110 @@ export function fullyPassed(testModule) {
  return true;
 }

+const log = (msg) => console.error(msg);
+
 export default class TestCacheReporter {
  onInit(vitest) {
    this.vitest = vitest;
    this.startedAt = Date.now();
  }

-  onTestRunEnd(testModules) {
+  onTestRunEnd(testModules, unhandledErrors = [], reason) {
    try {
-      this.record(testModules);
+      this.record(testModules, unhandledErrors, reason);
    } catch (err) {
-      console.error(`[test-cache] recorder failed, run unaffected: ${err?.stack ?? err}`);
+      log(`[test-cache] recorder failed, run unaffected: ${err?.stack ?? err}`);
    }
  }

-  record(testModules) {
+  record(testModules, unhandledErrors, reason) {
    const mode = core.mode();
    if (mode === 'off') return;
-    const state = globalThis.__civitaiTestCache ?? { hits: new Set(), sampled: new Set(), skipped: [], total: testModules.length };
+    const state = globalThis.__civitaiTestCache ?? {
+      hits: new Set(),
+      sampled: new Set(),
+      skipped: [],
+      startedAt: this.startedAt,
+    };
    const root = this.vitest.config.root;
-    const filtered = Boolean(this.vitest.config.testNamePattern);
    const dir = core.cacheDir(root);
    mkdirSync(dir, { recursive: true });
-    const fingerprint = core.makeFingerprinter(root);
-    const salt = core.globalSalt(root, this.vitest.version, fingerprint);

-    const rows = [];
-    for (const m of testModules) {
-      const project = m.project.name;
-      const testRel = core.toRel(m.moduleId, root);
+    const rows = testModules.map((m) => {
      const d = m.diagnostic();
-      const row = {
-        file: testRel,
-        project,
+      const passed = fullyPassed(m);
+      const wasHit = state.hits.has(`${m.project.name}\0${m.moduleId}`);
+      return {
+        m,
+        file: core.toRel(m.moduleId, root),
+        project: m.project.name,
        ms: (d.prepareDuration ?? 0) + (d.setupDuration ?? 0) + (d.collectDuration ?? 0) + (d.duration ?? 0),
-        passed: fullyPassed(m),
-        wasHit: state.hits.has(`${project}\0${m.moduleId}`),
+        passed,
+        wasHit,
+        falseSkip: wasHit && !passed,
        why: null,
      };
-      rows.push(row);
-      if (row.wasHit && !row.passed) row.falseSkip = true;
+    });

-      const reads = m.meta()?.testCacheReads;
-      // Whichever environment transformed the file. A `// @vitest-environment happy-dom` test goes
-      // through the web transform, so its modules are in `client`, not `ssr` — measured: such a
-      // file was absent from the ssr graph and fell out of the cache entirely.
-      const graph = Object.values(m.project.vite?.environments ?? {})
-        .map((env) => env.moduleGraph)
-        .find((g) => g?.getModuleById(m.moduleId));
-      let source = null;
-      try {
-        source = readFileSync(join(root, testRel), 'utf8');
-      } catch {
-        /* handled below */
-      }
-
-      if (!project.startsWith('unit')) row.why = 'not a unit project';
-      else if (!row.passed) row.why = 'did not fully pass';
-      else if (filtered) row.why = 'name-filtered run';
-      else if (source === null) row.why = 'test file unreadable';
-      else if (core.alwaysRuns(source)) row.why = 'spawns / computed import / glob';
-      // No tracker, no record: without it, a read made by a helper module is invisible.
-      else if (!Array.isArray(reads)) row.why = 'file reads not tracked';
-      else if (!graph) row.why = 'no ssr module graph';
-      if (row.why) continue;
-
-      const entries = new Set();
-      const own = core.closureOf(graph, m.moduleId);
-      if (own === null) {
-        row.why = 'test file not in module graph';
-        continue;
-      }
-      for (const id of own) entries.add(core.toRel(id, root));
-      let setupMissing = false;
-      for (const setup of m.project.config.setupFiles ?? []) {
-        const s = core.closureOf(graph, setup);
-        if (s === null) {
-          setupMissing = true;
-          break;
+    // The tripwire FIRST, before anything that can throw: a failure while recording must not be
+    // what stops a false skip from being reported.
+    const falseSkips = rows.filter((r) => r.falseSkip);
+    if (falseSkips.length) {
+      core.writeTripped(dir, {
+        at: new Date().toISOString(),
+        root,
+        mode,
+        falseSkips: falseSkips.map((r) => r.file),
+      });
+      for (const r of falseSkips) {
+        try {
+          core.forget(dir, r.project, r.file);
+        } catch (err) {
+          log(`[test-cache] could not delete records for ${r.file}: ${err?.message ?? err}`);
        }
-        entries.add(core.toRel(setup, root));
-        for (const id of s) entries.add(core.toRel(id, root));
      }
-      if (setupMissing) {
-        row.why = 'setup file not in module graph';
-        continue;
-      }
-      for (const p of reads) entries.add(core.toRel(p, root));
-
-      const kept = [...entries].filter((rel) => !core.isCoveredElsewhere(rel));
-      const key = core.keyFor({ salt, project, testRel, entries: kept, fingerprint });
-      core.writeRecord(dir, project, testRel, { key, entries: kept, at: new Date().toISOString(), ms: row.ms });
-      row.recorded = true;
+      log(
+        `[test-cache] 🔴 FALSE SKIP — the cache predicted these would pass and they did not:\n` +
+          falseSkips.map((r) => `    ${r.file}`).join('\n') +
+          `\n  Their records are deleted and the cache is TRIPPED: every run executes in full until ` +
+          `${core.trippedPath(dir)} is removed.`
+      );
    }

-    const falseSkips = rows.filter((r) => r.falseSkip).map((r) => r.file);
-    if (falseSkips.length) {
-      writeFileSync(
-        core.trippedPath(dir),
-        JSON.stringify({ at: new Date().toISOString(), root, mode, falseSkips }, null, 2)
-      );
+    // An unhandled error fails the run without failing any module, so no module's "passed" can be
+    // trusted — measured by review: a leaked rejection left its file `passed`, got recorded, and the
+    // next run skipped it green. Same for an interrupted run.
+    const runBlock =
+      unhandledErrors?.length > 0
+        ? 'unhandled errors in the run'
+        : reason === 'interrupted'
+          ? 'run interrupted'
+          : this.vitest.config.testNamePattern
+            ? 'name-filtered run'
+            : null;
+
+    const fingerprint = core.makeFingerprinter(root);
+    const salt = core.globalSalt(root, this.vitest.version, fingerprint);
+    const opaqueMemo = new Map();
+    const opaqueSource = (rel) => {
+      if (!opaqueMemo.has(rel)) {
+        let bad = false;
+        try {
+          bad = core.alwaysRuns(readFileSync(join(root, rel), 'utf8'));
+        } catch {
+          /* unreadable: its fingerprint records that */
+        }
+        opaqueMemo.set(rel, bad);
+      }
+      return opaqueMemo.get(rel);
+    };
+
+    for (const row of rows) {
+      try {
+        this.recordOne(row, { root, dir, fingerprint, salt, runBlock, since: state.startedAt, opaqueSource });
+      } catch (err) {
+        row.why = `record failed: ${err?.code ?? err?.message ?? err}`;
+      }
    }

    const skipped = state.skipped?.length ?? 0;
@@ -137,7 +140,8 @@ export default class TestCacheReporter {
      at: new Date().toISOString(),
      mode,
      root,
-      filtered,
+      bailed: state.bailed ?? null,
+      runBlock,
      total: rows.length + skipped,
      ran: rows.length,
      skipped,
@@ -145,9 +149,9 @@ export default class TestCacheReporter {
      recorded: rows.filter((r) => r.recorded).length,
      ranMs: Math.round(rows.reduce((n, r) => n + r.ms, 0)),
      hitMs: Math.round(rows.filter((r) => r.wasHit).reduce((n, r) => n + r.ms, 0)),
-      falseSkips,
-      notRecorded: tally(rows.filter((r) => r.why).map((r) => r.why)),
-      wallMs: Date.now() - this.startedAt,
+      falseSkips: falseSkips.map((r) => r.file),
+      notRecorded: tally(rows.filter((r) => r.why).map((r) => r.why.replace(/:.*$/, ''))),
+      wallMs: Date.now() - (state.startedAt ?? this.startedAt),
    };
    appendFileSync(join(dir, 'ledger.jsonl'), JSON.stringify(summary) + '\n');

@@ -155,17 +159,68 @@ export default class TestCacheReporter {
      mode === 'shadow'
        ? `${summary.hitsRan} unchanged file(s) ran (shadow mode skips nothing)`
        : `${summary.hitsRan} unchanged file(s) re-run to verify`;
-    console.log(
+    log(
      `[test-cache] ${summary.total} test files: ${summary.ran} ran, ${skipped} skipped as unchanged ` +
        `since they last passed. ${verify}; false skips: ${falseSkips.length}.`
    );
-    if (falseSkips.length) {
-      console.error(
-        `[test-cache] 🔴 FALSE SKIP — the cache predicted these would pass and they did not:\n` +
-          falseSkips.map((f) => `    ${f}`).join('\n') +
-          `\n  The cache is now TRIPPED: every run executes in full until ${core.trippedPath(dir)} is removed.`
-      );
+  }
+
+  recordOne(row, { root, dir, fingerprint, salt, runBlock, since, opaqueSource }) {
+    const { m, project, file: testRel } = row;
+    if (!project.startsWith('unit')) return void (row.why = 'not a unit project');
+    if (!row.passed) return void (row.why = 'did not fully pass');
+    if (runBlock) return void (row.why = runBlock);
+    if (testRel === null) return void (row.why = 'test file outside the repo');
+    const reads = m.meta()?.testCacheReads;
+    // No tracker, no record: without it, a read made by a helper module is invisible.
+    if (!Array.isArray(reads)) return void (row.why = 'file reads not tracked');
+
+    // Whichever environment transformed the file. A `// @vitest-environment happy-dom` test goes
+    // through the web transform, so its modules are in `client`, not `ssr` — measured: such a file
+    // was absent from the ssr graph and fell out of the cache entirely.
+    const graph = Object.values(m.project.vite?.environments ?? {})
+      .map((env) => env.moduleGraph)
+      .find((g) => g?.getModuleById(m.moduleId));
+    if (!graph) return void (row.why = 'test file not in module graph');
+
+    const ids = new Set(core.closureOf(graph, m.moduleId));
+    for (const setup of m.project.config.setupFiles ?? []) {
+      // The fs tracker is instrumentation, not an input: its closure (this cache's own code, which
+      // imports child_process) is covered by the salt. Walking it marked EVERY test as reaching a
+      // child process and recorded nothing at all — measured, which made every control vacuous.
+      if (core.toRel(setup, root)?.startsWith('scripts/test-cache/')) continue;
+      const s = core.closureOf(graph, setup);
+      if (s === null) return void (row.why = 'setup file not in module graph');
+      ids.add(setup);
+      for (const id of s) ids.add(id);
    }
+    // A spawned process or worker reads what it likes, and a computed import is invisible to the
+    // graph — anywhere in the closure, not only in the test file. Measured by review: a helper's
+    // `import(/* @vite-ignore */ file)` loaded a module the key never saw.
+    if (core.reachesOpaqueBuiltin(ids)) return void (row.why = 'reaches child_process / worker_threads');
+
+    const entries = new Set([testRel]);
+    for (const id of ids) entries.add(core.toRel(id, root));
+    for (const p of reads) entries.add(core.toRel(p, root));
+    const kept = [...entries].filter((rel) => !core.isCoveredElsewhere(rel));
+
+    for (const rel of kept) {
+      if (/\.[cm]?[jt]sx?$/.test(rel) && opaqueSource(rel)) {
+        return void (row.why = `spawns / computed import / glob in ${rel}`);
+      }
+    }
+    // The key is computed from disk NOW, at the end of the run. An input modified after the run
+    // started is not what ran — recording it would certify the edited version as passing.
+    this.movedMemo ??= new Map();
+    const moved = kept.find((rel) => {
+      if (!this.movedMemo.has(rel)) this.movedMemo.set(rel, core.changedSince(root, rel, since - 2000));
+      return this.movedMemo.get(rel);
+    });
+    if (moved) return void (row.why = `changed during the run: ${moved}`);
+
+    const key = core.keyFor({ salt, project, testRel, entries: kept, fingerprint });
+    core.writeRecord(dir, project, testRel, { key, entries: kept, at: new Date().toISOString(), ms: row.ms });
+    row.recorded = true;
  }
 }

@@ -5,8 +5,9 @@
 *
 * A file is skippable when one of its recorded passes still matches: the record lists every path
 * the file depended on last time it passed, and re-fingerprinting those paths now reproduces the
- * recorded key. Reusing the OLD dependency list is sound, because the only way to gain a dependency
- * is to edit a file already in the list — and that edit changes the key.
+ * recorded key. Reusing the OLD list is sound for static imports and tracked file reads, because
+ * gaining one means editing a file already in the list. It is NOT sound for a computed dynamic
+ * import or a spawned process; the reporter refuses to record any file whose graph can reach one.
 *
 * In `on` mode a random sample of the hits runs anyway. If a sampled hit then fails, the key missed
 * something; the reporter trips the cache off and says so. The sample is the standing check that
@@ -17,7 +18,11 @@ import { BaseSequencer } from 'vitest/node';

 import * as core from './core.mjs';

-const SAMPLE_RATE = Number(process.env.CIVITAI_TEST_CACHE_SAMPLE ?? 0.05);
+const parsedRate = Number(process.env.CIVITAI_TEST_CACHE_SAMPLE);
+// A malformed rate must not become NaN: `Math.random() < NaN` is never true, which would switch the
+// sampling — the tripwire's only input in `on` mode — silently off.
+const SAMPLE_RATE =
+  process.env.CIVITAI_TEST_CACHE_SAMPLE !== undefined && Number.isFinite(parsedRate) ? parsedRate : 0.05;

 export const runKey = (spec) => `${spec.project.name}\0${spec.moduleId}`;

@@ -25,16 +30,31 @@ export default class TestCacheSequencer extends BaseSequencer {
  async sort(files) {
    const state = {
      mode: core.mode(),
+      // Recorded inputs modified after this instant were not what ran; the reporter refuses them.
+      startedAt: Date.now(),
      total: files.length,
      skipped: [],
      hits: new Set(),
      sampled: new Set(),
      tripped: null,
+      bailed: null,
    };
    globalThis.__civitaiTestCache = state;

-    // A name filter runs part of each file, so a partial run must not be traded for a skip either.
-    if (state.mode === 'off' || this.ctx.config.testNamePattern) return super.sort(files);
+    if (state.mode === 'off') return super.sort(files);
+    // A name filter runs part of each file, so a partial run must not be traded for a skip.
+    if (this.ctx.config.testNamePattern) state.bailed = 'name filter';
+    // A run that names files asked for those files. Only a whole-suite run is the cache's to trim.
+    else if (this.ctx.filenamePattern?.length && !process.env.CIVITAI_TEST_CACHE_ALLOW_FILTERS)
+      state.bailed = 'file filter';
+    // Without the reporter nothing records and nothing checks the sample, so skipping would be
+    // unobserved. It is loaded by the queue on the command line; a hand-exported env var is not.
+    else if (!(this.ctx.reporters ?? []).some((r) => r?.constructor?.name === 'TestCacheReporter'))
+      state.bailed = 'cache reporter not loaded';
+    if (state.bailed) {
+      console.error(`[test-cache] not skipping anything: ${state.bailed}.`);
+      return super.sort(files);
+    }

    try {
      const root = this.ctx.config.root;
@@ -67,8 +87,10 @@ export default class TestCacheSequencer extends BaseSequencer {
        state.skipped.push(testRel);
      }

+      // stderr, not stdout: a caller's own `--reporter=json` with no outputFile writes its JSON to
+      // stdout, and a line from here would corrupt it.
      if (state.mode === 'on') {
-        console.log(
+        console.error(
          state.tripped
            ? `[test-cache] TRIPPED since ${state.tripped.at} — running everything. ` +
                `Delete ${core.trippedPath(dir)} once the cause is understood.`
@@ -180,9 +180,10 @@ const unitTestConfig = {
  exclude: ['node_modules', 'tests/**/*'], // Exclude Playwright tests
  // The fs tracker records which files each test reads, for the result cache. It is loaded only
  // when the queue turned the cache on (CIVITAI_TEST_CACHE) and never in CI — see scripts/test-cache.
+  // Tracker FIRST, so reads made while the main setup file loads are recorded too.
  setupFiles: [
-    'src/__tests__/setup.ts',
    ...(testCacheMode() !== 'off' ? ['scripts/test-cache/fs-tracker.mjs'] : []),
+    'src/__tests__/setup.ts',
  ],
  // Several unit tests cold-`await import(...)` a large Next API-page / service
  // module graph (mocked I/O, but a real ~9–16s TS transform). With the suite's