test: prune abandoned test-run tmp directories at run setup (#1834)

* test: prune abandoned test-run tmp directories at run setup

A run killed before its teardown (tool-timeout SIGKILL, OOM, cancelled job)
left /tmp/agent-device-test-run-<pid>-* behind, and check:tmpdir-leaks — which
runs after test:unit in check:unit — flagged every dead-pid directory it
found. It could not tell this run's leak from a historical one, so one killed
run made every later, otherwise-green gate on the host fail.

Both TMPDIR redirection entry points (the Vitest global setup and the
node --test wrapper) now prune dead-pid run directories before creating their
own, printing one [tmpdir] line when they did; the post-run check keeps its
semantics and can now only ever name the run that just finished. Live owners
(a concurrent run in another worktree) are never touched.

The root/prefix constants move into check-tmpdir-leaks-model.ts, next to the
liveness classification, so the setup can import the prune without a cycle.

* test(tmpdir): a run directory is live while any process still holds it as TMPDIR, not only while its owner runs

Review (P1): owner-pid liveness alone would prune a directory out from under
the orphaned children of a SIGKILLed run — the node --test chain, Vitest forks,
or a daemon a test spawned all keep running with that TMPDIR. The liveness
model now reads every process's TMPDIR (ps -E on macOS, /proc/<pid>/environ
on Linux) and treats a run directory as live while its owner pid is alive OR
any process's TMPDIR points into it; both the prune and the post-run leak
check use it. Regression: a wrapped probe spawns a detached long-lived child,
only the wrapper is SIGKILLed, the next prune preserves the directory; after
every consumer exits, the next prune removes it. Planted red with owner-only
liveness: the orphaned directory is pruned.
This commit is contained in:
Michał Pierzchała
2026-08-18 17:48:12 +02:00
committed by GitHub
parent 423927fdd8
commit 0fb38f1da2
8 changed files with 478 additions and 38 deletions
+12
View File
@@ -78,6 +78,18 @@ cleanup for a directory they created — that's the global teardown's job, and p
already existed for other reasons should stay (it's the fallback global sweep that's new, not a
replacement for tests being tidy).
A run killed before its teardown (a tool timeout's SIGKILL, OOM, a cancelled job) leaves its
`/tmp/agent-device-test-run-<pid>-*` directory behind. The next run on the host prunes every such
directory that is genuinely abandoned (`pruneAbandonedRunDirectories`, called by both the Vitest
global setup and the `node --test` wrapper) and prints one `[tmpdir] pruned …` line, so
`check:tmpdir-leaks` after `test:unit` can only ever name the run that just finished. Abandoned
means nobody owns it **and nobody uses it**: the owner pid in the name is dead and no live process
has a `TMPDIR` inside it (read from `ps -E` on macOS, `/proc/<pid>/environ` on Linux). That second
half matters because a SIGKILL of the wrapper or Vitest main process leaves its `node --test` chain,
forked workers, and any daemon a test spawned running with that `TMPDIR` — they keep the directory
until the last of them exits. A concurrent run in another worktree is live by both tests. If the
check fails, the leak is this run's: a teardown that did not execute, not history.
Keep tests behavioral. Do not assert shapes or cases TypeScript already proves.
A test added as a regression pin must be shown to fail without the change it pins — vacuity is the
+98 -7
View File
@@ -3,7 +3,13 @@ import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { test } from 'node:test';
import { findLeakedRunDirectories } from './check-tmpdir-leaks-model.ts';
import {
findLeakedRunDirectories,
pruneAbandonedRunDirectories,
runDirectoryNameOf,
} from './check-tmpdir-leaks-model.ts';
const NONE: ReadonlySet<string> = new Set();
function withScratchRoot(fn: (root: string) => void): void {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'check-tmpdir-leaks-model-test-'));
@@ -17,7 +23,10 @@ function withScratchRoot(fn: (root: string) => void): void {
test('a directory owned by a still-running process is not reported as a leak', () => {
withScratchRoot((root) => {
fs.mkdirSync(path.join(root, 'agent-device-test-run-4242-abcdef'));
const leaks = findLeakedRunDirectories(root, (pid) => pid === 4242);
const leaks = findLeakedRunDirectories(root, {
isAlive: (pid) => pid === 4242,
consumers: NONE,
});
assert.deepEqual(leaks, []);
});
});
@@ -25,7 +34,7 @@ test('a directory owned by a still-running process is not reported as a leak', (
test('a directory whose owning process has exited is reported as a leak', () => {
withScratchRoot((root) => {
fs.mkdirSync(path.join(root, 'agent-device-test-run-4242-abcdef'));
const leaks = findLeakedRunDirectories(root, () => false);
const leaks = findLeakedRunDirectories(root, { isAlive: () => false, consumers: NONE });
assert.deepEqual(leaks, ['agent-device-test-run-4242-abcdef']);
});
});
@@ -38,7 +47,10 @@ test('a concurrent run from another worktree does not fail the check for this on
fs.mkdirSync(path.join(root, 'agent-device-test-run-1111-aaaaaa'));
fs.mkdirSync(path.join(root, 'agent-device-test-run-2222-bbbbbb'));
const alivePids = new Set([1111, 2222]);
const leaks = findLeakedRunDirectories(root, (pid) => alivePids.has(pid));
const leaks = findLeakedRunDirectories(root, {
isAlive: (pid) => alivePids.has(pid),
consumers: NONE,
});
assert.deepEqual(leaks, []);
});
});
@@ -47,7 +59,10 @@ test('a mix of active and abandoned directories reports only the abandoned one',
withScratchRoot((root) => {
fs.mkdirSync(path.join(root, 'agent-device-test-run-1111-aaaaaa')); // alive
fs.mkdirSync(path.join(root, 'agent-device-test-run-3333-cccccc')); // exited
const leaks = findLeakedRunDirectories(root, (pid) => pid === 1111);
const leaks = findLeakedRunDirectories(root, {
isAlive: (pid) => pid === 1111,
consumers: NONE,
});
assert.deepEqual(leaks, ['agent-device-test-run-3333-cccccc']);
});
});
@@ -55,7 +70,7 @@ test('a mix of active and abandoned directories reports only the abandoned one',
test('a directory with no parseable pid is conservatively reported as a leak', () => {
withScratchRoot((root) => {
fs.mkdirSync(path.join(root, 'agent-device-test-run-not-a-pid'));
const leaks = findLeakedRunDirectories(root, () => true);
const leaks = findLeakedRunDirectories(root, { isAlive: () => true, consumers: NONE });
assert.deepEqual(leaks, ['agent-device-test-run-not-a-pid']);
});
});
@@ -64,7 +79,83 @@ test('non-matching directories and files are ignored', () => {
withScratchRoot((root) => {
fs.mkdirSync(path.join(root, 'unrelated-directory'));
fs.writeFileSync(path.join(root, 'agent-device-test-run-4242-loose-file'), '');
const leaks = findLeakedRunDirectories(root, () => false);
const leaks = findLeakedRunDirectories(root, { isAlive: () => false, consumers: NONE });
assert.deepEqual(leaks, []);
});
});
test('pruning removes only abandoned directories and returns their names', () => {
withScratchRoot((root) => {
fs.mkdirSync(path.join(root, 'agent-device-test-run-1111-aaaaaa')); // alive
fs.mkdirSync(path.join(root, 'agent-device-test-run-3333-cccccc', 'nested'), {
recursive: true,
}); // exited, non-empty
fs.mkdirSync(path.join(root, 'agent-device-test-run-not-a-pid')); // no owner
fs.mkdirSync(path.join(root, 'unrelated-directory'));
const pruned = pruneAbandonedRunDirectories(root, {
isAlive: (pid) => pid === 1111,
consumers: NONE,
});
assert.deepEqual(pruned.sort(), [
'agent-device-test-run-3333-cccccc',
'agent-device-test-run-not-a-pid',
]);
assert.deepEqual(fs.readdirSync(root).sort(), [
'agent-device-test-run-1111-aaaaaa',
'unrelated-directory',
]);
// Once pruned, the post-run check has nothing historical left to report.
assert.deepEqual(
findLeakedRunDirectories(root, { isAlive: (pid) => pid === 1111, consumers: NONE }),
[],
);
});
});
test('pruning nothing is a no-op that reports nothing', () => {
withScratchRoot((root) => {
fs.mkdirSync(path.join(root, 'agent-device-test-run-1111-aaaaaa'));
assert.deepEqual(
pruneAbandonedRunDirectories(root, { isAlive: () => true, consumers: NONE }),
[],
);
assert.deepEqual(fs.readdirSync(root), ['agent-device-test-run-1111-aaaaaa']);
});
});
test('a directory whose owner is dead but which some live process still holds as TMPDIR is not a leak', () => {
withScratchRoot((root) => {
// The motivating case: the wrapper/vitest owner was SIGKILLed, its node --test chain or
// forked workers (or a daemon a test spawned) are still running with TMPDIR inside the
// directory. Nobody may prune it until the last of them exits.
fs.mkdirSync(path.join(root, 'agent-device-test-run-4242-orphaned'));
fs.mkdirSync(path.join(root, 'agent-device-test-run-5555-finished'));
const consumers = new Set(['agent-device-test-run-4242-orphaned']);
assert.deepEqual(findLeakedRunDirectories(root, { isAlive: () => false, consumers }), [
'agent-device-test-run-5555-finished',
]);
assert.deepEqual(pruneAbandonedRunDirectories(root, { isAlive: () => false, consumers }), [
'agent-device-test-run-5555-finished',
]);
assert.deepEqual(fs.readdirSync(root), ['agent-device-test-run-4242-orphaned']);
// Once the consumers are gone it is an ordinary abandoned directory.
assert.deepEqual(
pruneAbandonedRunDirectories(root, { isAlive: () => false, consumers: NONE }),
['agent-device-test-run-4242-orphaned'],
);
assert.deepEqual(fs.readdirSync(root), []);
});
});
test('a consumer is identified by the run directory its TMPDIR sits inside, at any depth', () => {
assert.equal(
runDirectoryNameOf('/tmp/agent-device-test-run-123-abc'),
'agent-device-test-run-123-abc',
);
assert.equal(
runDirectoryNameOf('/tmp/agent-device-test-run-123-abc/nested/deeper'),
'agent-device-test-run-123-abc',
);
assert.equal(runDirectoryNameOf('/tmp/other-123-abc'), undefined);
assert.equal(runDirectoryNameOf('/var/folders/x/T/agent-device-test-run-1-a'), undefined);
});
+130 -5
View File
@@ -1,5 +1,23 @@
import fs from 'node:fs';
import { TEST_RUN_TMP_PREFIX } from './vitest-tmpdir-global-setup.ts';
import path from 'node:path';
import { runCmdSync } from '../src/utils/exec.ts';
// Every test run (Vitest via scripts/vitest-tmpdir-global-setup.ts, node --test
// via scripts/node-test-tmpdir.ts) redirects TMPDIR into one disposable,
// pid-tagged directory under this root and removes it at teardown.
//
// Rooted at /tmp rather than nested inside the current os.tmpdir(): macOS's
// per-user TMPDIR (/var/folders/.../T/) is already close to the 104-byte
// sun_path limit AF_UNIX sockets need, and tests that bind real sockets
// (e.g. runner-usbmux.test.ts) started hitting EINVAL once nested one level
// deeper. /tmp is short enough to leave headroom for those.
//
// Both redirection mechanisms and check-tmpdir-leaks.ts import these two from
// here rather than recomputing them, so they can't drift onto different
// directories (os.tmpdir() != /tmp on macOS, where TMPDIR is a deep per-user
// path).
export const TEST_RUN_TMP_ROOT = '/tmp';
export const TEST_RUN_TMP_PREFIX = 'agent-device-test-run-';
const PID_SUFFIX = new RegExp(`^${TEST_RUN_TMP_PREFIX}(\\d+)-`);
@@ -14,18 +32,91 @@ function isProcessAlive(pid: number): boolean {
}
/**
* Directories owned by a still-running process are a concurrent vitest run
* (e.g. another worktree), not a leak — only report the ones whose owning
* process has already exited without cleaning up after itself.
* The run directories some live process is still using: every process whose TMPDIR points
* into one. That is the ownership signal every consumer actually carries — the run's own
* setup exports it and every child (Vitest forks, the node --test chain, daemons a test
* spawned) inherits it — so a run whose owner was SIGKILLed while its children kept running
* is still "in use" until the last of them exits. Read from `ps -E` on macOS (environment
* is shown for the caller's own processes) and /proc/<pid>/environ on Linux; on either, a
* process this user cannot inspect contributes nothing, and its directory is then judged by
* its owner pid alone.
*/
export function liveRunDirectoryConsumers(): ReadonlySet<string> {
const consumers = new Set<string>();
for (const value of readAllProcessTmpdirs()) {
const name = runDirectoryNameOf(value);
if (name !== undefined) consumers.add(name);
}
return consumers;
}
/** `/tmp/agent-device-test-run-123-abc/nested` → `agent-device-test-run-123-abc`; else undefined. */
export function runDirectoryNameOf(tmpdir: string): string | undefined {
const rootPrefix = `${TEST_RUN_TMP_ROOT}/`;
if (!tmpdir.startsWith(rootPrefix)) return undefined;
const name = tmpdir.slice(rootPrefix.length).split('/')[0] ?? '';
return name.startsWith(TEST_RUN_TMP_PREFIX) ? name : undefined;
}
function readAllProcessTmpdirs(): string[] {
return process.platform === 'linux' ? readProcTmpdirs() : readPsTmpdirs();
}
function readProcTmpdirs(): string[] {
return fs
.readdirSync('/proc')
.filter((entry) => /^\d+$/.test(entry))
.flatMap((pid) => tmpdirsOfEnviron(readEnvironOrEmpty(pid)));
}
/** Another user's process, or one that exited mid-scan, contributes nothing. */
function readEnvironOrEmpty(pid: string): string {
try {
return fs.readFileSync(`/proc/${pid}/environ`, 'latin1');
} catch {
return '';
}
}
function tmpdirsOfEnviron(environ: string): string[] {
return environ
.split('\0')
.filter((pair) => pair.startsWith('TMPDIR='))
.map((pair) => pair.slice('TMPDIR='.length));
}
// macOS (and other BSDs): -E appends the environment to each command line. Every process's
// environment is a few MB on a busy host — well past spawnSync's 1 MB default.
function readPsTmpdirs(): string[] {
const listing = runCmdSync('ps', ['-axEww', '-o', 'command='], {
allowFailure: true,
maxBuffer: 64 * 1024 * 1024,
});
if (listing.exitCode !== 0) return [];
return [...listing.stdout.matchAll(/(?:^|\s)TMPDIR=(\S+)/g)].map((match) => match[1] as string);
}
export type RunDirectoryLiveness = Readonly<{
isAlive?: (pid: number) => boolean;
consumers?: ReadonlySet<string>;
}>;
/**
* A run directory is live while its owning process (the pid in its name) runs, OR while any
* process still holds it as TMPDIR — a concurrent run in another worktree, or the orphaned
* children of a killed run. Only the rest are leaks: nobody owns them and nobody uses them.
*/
export function findLeakedRunDirectories(
root: string,
isAlive: (pid: number) => boolean = isProcessAlive,
liveness: RunDirectoryLiveness = {},
): string[] {
const isAlive = liveness.isAlive ?? isProcessAlive;
const consumers = liveness.consumers ?? liveRunDirectoryConsumers();
return fs
.readdirSync(root, { withFileTypes: true })
.filter((entry) => entry.isDirectory() && entry.name.startsWith(TEST_RUN_TMP_PREFIX))
.filter((entry) => {
if (consumers.has(entry.name)) return false;
const match = PID_SUFFIX.exec(entry.name);
// No parseable pid means it didn't come from setup() as written — treat it as a leak.
if (!match) return true;
@@ -33,3 +124,37 @@ export function findLeakedRunDirectories(
})
.map((entry) => entry.name);
}
/**
* Removes the run directories an earlier, already-exited run left behind and
* returns their names. A run's setup calls this before creating its own
* directory, so the post-run leak check (check-tmpdir-leaks.ts) can only ever
* report the run that just finished: a directory abandoned by an earlier run
* that was killed before its teardown (SIGKILL on a tool timeout, OOM, a
* cancelled CI job) is by construction the same thing that teardown would have
* removed, and leaving it in place made every later, otherwise-green gate on
* the host fail for a run it never ran. Live owners are never touched, so a
* concurrent run in another worktree keeps its directory.
*/
export function pruneAbandonedRunDirectories(
root: string,
liveness: RunDirectoryLiveness = {},
): string[] {
const abandoned = findLeakedRunDirectories(root, liveness);
for (const name of abandoned) {
fs.rmSync(path.join(root, name), { recursive: true, force: true });
}
return abandoned;
}
/**
* One stderr line, only when something was pruned: an earlier run on this
* host died before its teardown, which the operator should know (a tool
* timeout killed it, say) without it being a failure of this run.
*/
export function reportPrunedRunDirectories(pruned: readonly string[]): void {
if (pruned.length === 0) return;
process.stderr.write(
`[tmpdir] pruned ${pruned.length} abandoned ${TEST_RUN_TMP_PREFIX}* director${pruned.length === 1 ? 'y' : 'ies'} left by an earlier killed run\n`,
);
}
+10 -8
View File
@@ -1,19 +1,21 @@
// Fails if any agent-device-test-run-* directory under TEST_RUN_TMP_ROOT is
// abandoned — its owning process has exited without running its cleanup
// (crash, OOM, timeout kill). Directories owned by a still-running process
// are left alone: a concurrent `vitest run` (or node --test lane, wrapped by
// scripts/node-test-tmpdir.ts) in another worktree keeps its own directory
// present until its own teardown, which is not a leak. See
// check-tmpdir-leaks-model.ts for the liveness check.
// (crash, OOM, timeout kill) AND no live process still holds it as TMPDIR.
// Directories owned by a still-running process are left alone: a concurrent
// `vitest run` (or node --test lane, wrapped by scripts/node-test-tmpdir.ts) in
// another worktree keeps its own directory present until its own teardown,
// which is not a leak; so are the orphaned children of a killed run, until the
// last of them exits. See check-tmpdir-leaks-model.ts for the liveness model.
//
// Covers both redirection mechanisms sharing this root/prefix: Vitest's
// globalSetup/globalTeardown (scripts/vitest-tmpdir-global-setup.ts) and the
// node --test wrapper (scripts/node-test-tmpdir.ts, #1595) that every
// `node --test` package.json script now runs through.
// `node --test` package.json script now runs through. Both prune what an
// earlier killed run left behind before creating their own directory, so a
// failure here names the run that just finished — never a historical one.
import path from 'node:path';
import { findLeakedRunDirectories } from './check-tmpdir-leaks-model.ts';
import { TEST_RUN_TMP_ROOT } from './vitest-tmpdir-global-setup.ts';
import { TEST_RUN_TMP_ROOT, findLeakedRunDirectories } from './check-tmpdir-leaks-model.ts';
const leaks = findLeakedRunDirectories(TEST_RUN_TMP_ROOT);
+160 -2
View File
@@ -5,8 +5,13 @@ import os from 'node:os';
import path from 'node:path';
import { test } from 'node:test';
import { fileURLToPath } from 'node:url';
import { runCmd } from '../src/utils/exec.ts';
import { TEST_RUN_TMP_PREFIX, TEST_RUN_TMP_ROOT } from './vitest-tmpdir-global-setup.ts';
import { runCmd, runCmdBackground } from '../src/utils/exec.ts';
import {
liveRunDirectoryConsumers,
pruneAbandonedRunDirectories,
TEST_RUN_TMP_PREFIX,
TEST_RUN_TMP_ROOT,
} from './check-tmpdir-leaks-model.ts';
const REPOSITORY_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const WRAPPER = path.join(REPOSITORY_ROOT, 'scripts', 'node-test-tmpdir.ts');
@@ -250,3 +255,156 @@ test('every node --test package.json script routes through scripts/node-test-tmp
`NODE_TEST_WRAPPER_BYPASS_ALLOWLIST above with a reason if one must legitimately bypass it.`,
);
});
// INT32_MAX exceeds every platform's pid range (Linux pid_max caps at 2^22,
// macOS at 99999), so kill(pid, 0) is ESRCH by construction — an owner that
// is dead and can never be reused mid-test, unlike a freshly exited child's pid.
const NEVER_A_PID = 2_147_483_647;
test('the wrapper prunes a run directory abandoned by an earlier killed run and keeps a live one', async () => {
const stamp = crypto.randomUUID();
const abandoned = path.join(
TEST_RUN_TMP_ROOT,
`${TEST_RUN_TMP_PREFIX}${NEVER_A_PID}-planted-${stamp}`,
);
const live = path.join(
TEST_RUN_TMP_ROOT,
`${TEST_RUN_TMP_PREFIX}${process.pid}-planted-${stamp}`,
);
const evidenceRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'node-test-tmpdir-prune-'));
const probePath = writeProbe(path.join(evidenceRoot, 'child-tmpdir.txt'));
fs.mkdirSync(path.join(abandoned, 'nested'), { recursive: true });
fs.writeFileSync(path.join(abandoned, 'nested', 'leftover.txt'), 'from a killed run');
fs.mkdirSync(live);
try {
const result = await runCmd(
process.execPath,
['--experimental-strip-types', WRAPPER, '--experimental-strip-types', '--test', probePath],
{ cwd: REPOSITORY_ROOT, timeoutMs: 30_000 },
);
assert.equal(result.exitCode, 0, `probe run failed:\n${result.stdout}\n${result.stderr}`);
assert.equal(
fs.existsSync(abandoned),
false,
'the wrapper must prune the abandoned run directory',
);
assert.equal(
fs.existsSync(live),
true,
'the wrapper must never touch a live owners run directory',
);
} finally {
fs.rmSync(abandoned, { recursive: true, force: true });
fs.rmSync(live, { recursive: true, force: true });
fs.rmSync(probePath, { force: true });
fs.rmSync(evidenceRoot, { recursive: true, force: true });
}
});
test('a run whose owner alone was killed keeps its directory while a child still uses it, and loses it once the child exits', async () => {
// The motivating case for consumer-aware pruning: a tool timeout SIGKILLs the wrapper (the
// owner pid in the directory name) while something it started — here a detached child the
// probe spawned, the shape of a daemon a test brought up — is still running with TMPDIR
// pointing into the run directory. Owner-pid liveness alone would prune the directory out
// from under that child on the next run.
const evidenceRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'node-test-tmpdir-orphan-'));
const evidencePath = path.join(evidenceRoot, 'evidence.json');
const readyPath = path.join(evidenceRoot, 'ready');
const probeName = `node-test-tmpdir-probe-orphan-${process.pid}-${crypto.randomUUID()}.test.ts`;
const probePath = path.join(REPOSITORY_ROOT, 'scripts', probeName);
fs.writeFileSync(
probePath,
`import fs from 'node:fs';
import os from 'node:os';
import { test } from 'node:test';
import { runCmdDetached } from '../src/utils/exec.ts';
test('probe starts a long-lived detached child that inherits TMPDIR, then waits to be killed', async () => {
const childPid = runCmdDetached(process.execPath, ['-e', 'setTimeout(() => {}, 60_000)']);
fs.writeFileSync(
${JSON.stringify(evidencePath)},
JSON.stringify({ tmpdir: os.tmpdir(), childPid, probePid: process.pid, runnerPid: process.ppid }),
);
fs.writeFileSync(${JSON.stringify(readyPath)}, '');
await new Promise((resolve) => setTimeout(resolve, 60_000));
});
`,
);
let evidence:
| { tmpdir: string; childPid: number; probePid: number; runnerPid: number }
| undefined;
const consumersOf = (e: NonNullable<typeof evidence>) => [e.childPid, e.probePid, e.runnerPid];
const wrapper = runCmdBackground(
process.execPath,
['--experimental-strip-types', WRAPPER, '--experimental-strip-types', '--test', probePath],
{ cwd: REPOSITORY_ROOT, captureOutput: false, stdio: 'ignore', allowFailure: true },
);
try {
await waitFor(() => fs.existsSync(readyPath), 20_000, 'probe never reported ready');
evidence = JSON.parse(fs.readFileSync(evidencePath, 'utf8'));
const runDir = evidence!.tmpdir;
assert.equal(path.dirname(runDir), TEST_RUN_TMP_ROOT);
assert.equal(isAlive(evidence!.childPid), true, 'the detached child must be running');
// Kill only the owner. Its exit handler cannot run on SIGKILL, so the directory survives it.
process.kill(wrapper.child.pid!, 'SIGKILL');
await wrapper.wait;
assert.equal(fs.existsSync(runDir), true, 'SIGKILL of the owner leaves the directory behind');
assert.equal(isAlive(evidence!.childPid), true, 'the detached child outlives its owner');
// The next run's prune must see the child holding TMPDIR and leave the directory alone.
assert.deepEqual(
pruneAbandonedRunDirectories(TEST_RUN_TMP_ROOT).filter((name) => runDir.endsWith(name)),
[],
);
assert.equal(fs.existsSync(runDir), true, 'a directory with a live consumer is not pruned');
assert.equal(fs.existsSync(path.join(runDir)), true);
// Once every consumer is gone — the detached child AND the orphaned node --test chain,
// which holds the same TMPDIR — it is an ordinary abandoned directory.
for (const pid of consumersOf(evidence!)) if (isAlive(pid)) process.kill(pid, 'SIGKILL');
await waitFor(
() => !liveRunDirectoryConsumers().has(path.basename(runDir)),
20_000,
'the run directory still shows a live consumer after every child exited',
);
assert.deepEqual(
pruneAbandonedRunDirectories(TEST_RUN_TMP_ROOT).filter((name) => runDir.endsWith(name)),
[path.basename(runDir)],
);
assert.equal(fs.existsSync(runDir), false, 'with no owner and no consumer it is pruned');
} finally {
for (const pid of evidence ? consumersOf(evidence) : []) {
if (isAlive(pid)) process.kill(pid, 'SIGKILL');
}
if (wrapper.child.exitCode === null && wrapper.child.signalCode === null) {
wrapper.child.kill('SIGKILL');
}
if (evidence) fs.rmSync(evidence.tmpdir, { recursive: true, force: true });
fs.rmSync(probePath, { force: true });
fs.rmSync(evidenceRoot, { recursive: true, force: true });
}
});
function isAlive(pid: number): boolean {
try {
process.kill(pid, 0);
return true;
} catch {
return false;
}
}
async function waitFor(
condition: () => boolean,
timeoutMs: number,
message: string,
): Promise<void> {
const deadline = Date.now() + timeoutMs;
while (!condition()) {
if (Date.now() > deadline) throw new Error(message);
await new Promise((resolve) => setTimeout(resolve, 100));
}
}
+11 -4
View File
@@ -20,14 +20,20 @@
// process forwards to a still-running child (Ctrl-C locally, or a CI job
// cancellation): the signal handlers below call `process.exit()`, which
// triggers 'exit' synchronously before the process actually terminates.
// Only a SIGKILL against this wrapper itself bypasses all of that the same
// residual case check-tmpdir-leaks-model.ts already tolerates via its
// pid-liveness check, since it shares this directory's root and prefix.
// Only a SIGKILL against this wrapper itself bypasses all of that; the next
// run on the host prunes what such a kill left behind (see
// pruneAbandonedRunDirectories), since both lanes share this directory's root
// and prefix.
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { runCmdBackground } from '../src/utils/exec.ts';
import { TEST_RUN_TMP_PREFIX, TEST_RUN_TMP_ROOT } from './vitest-tmpdir-global-setup.ts';
import {
TEST_RUN_TMP_PREFIX,
TEST_RUN_TMP_ROOT,
pruneAbandonedRunDirectories,
reportPrunedRunDirectories,
} from './check-tmpdir-leaks-model.ts';
const forwardedArgs = process.argv.slice(2);
if (forwardedArgs.length === 0) {
@@ -36,6 +42,7 @@ if (forwardedArgs.length === 0) {
);
}
reportPrunedRunDirectories(pruneAbandonedRunDirectories(TEST_RUN_TMP_ROOT));
const testRunTmpDir = fs.mkdtempSync(
path.join(TEST_RUN_TMP_ROOT, `${TEST_RUN_TMP_PREFIX}${process.pid}-`),
);
+48 -1
View File
@@ -1,11 +1,12 @@
import assert from 'node:assert/strict';
import crypto from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { test } from 'node:test';
import { fileURLToPath } from 'node:url';
import { runCmd } from '../src/utils/exec.ts';
import { TEST_RUN_TMP_PREFIX, TEST_RUN_TMP_ROOT } from './vitest-tmpdir-global-setup.ts';
import { TEST_RUN_TMP_PREFIX, TEST_RUN_TMP_ROOT } from './check-tmpdir-leaks-model.ts';
const REPOSITORY_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
@@ -66,3 +67,49 @@ test('worker inherits the run-owned temp directory', () => {
fs.rmSync(evidenceRoot, { recursive: true, force: true });
}
});
// INT32_MAX exceeds every platform's pid range (Linux pid_max caps at 2^22,
// macOS at 99999), so kill(pid, 0) is ESRCH by construction — an owner that
// is dead and can never be reused mid-test, unlike a freshly exited child's pid.
const NEVER_A_PID = 2_147_483_647;
test('global setup prunes a run directory abandoned by an earlier killed run and keeps a live one', async () => {
const stamp = crypto.randomUUID();
const abandoned = path.join(
TEST_RUN_TMP_ROOT,
`${TEST_RUN_TMP_PREFIX}${NEVER_A_PID}-planted-${stamp}`,
);
// Owned by this test process, which is alive for the whole nested run: the
// same shape as a concurrent run in another worktree, and must survive.
const live = path.join(
TEST_RUN_TMP_ROOT,
`${TEST_RUN_TMP_PREFIX}${process.pid}-planted-${stamp}`,
);
const probeName = `vitest-tmpdir-prune-probe-${process.pid}-${stamp}.test.ts`;
const probePath = path.join(REPOSITORY_ROOT, 'src', '__tests__', probeName);
fs.mkdirSync(path.join(abandoned, 'nested'), { recursive: true });
fs.writeFileSync(path.join(abandoned, 'nested', 'leftover.txt'), 'from a killed run');
fs.mkdirSync(live);
fs.writeFileSync(
probePath,
`import { test } from 'vitest';
test('noop probe: the run itself is the subject', () => {});
`,
);
try {
const result = await runCmd(
path.join(REPOSITORY_ROOT, 'node_modules', '.bin', 'vitest'),
['run', '--project', 'unit-core', probePath],
{ cwd: REPOSITORY_ROOT, timeoutMs: 30_000 },
);
assert.equal(result.exitCode, 0, `probe run failed:\n${result.stdout}\n${result.stderr}`);
assert.equal(fs.existsSync(abandoned), false, 'setup must prune the abandoned run directory');
assert.equal(fs.existsSync(live), true, 'setup must never touch a live owners run directory');
} finally {
fs.rmSync(abandoned, { recursive: true, force: true });
fs.rmSync(live, { recursive: true, force: true });
fs.rmSync(probePath, { force: true });
}
});
+9 -11
View File
@@ -1,6 +1,12 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import {
TEST_RUN_TMP_PREFIX,
TEST_RUN_TMP_ROOT,
pruneAbandonedRunDirectories,
reportPrunedRunDirectories,
} from './check-tmpdir-leaks-model.ts';
// os.tmpdir() reads TMPDIR on every call, so redirecting it here covers every
// mkdtemp call site — test and production — without touching any of them.
@@ -12,17 +18,8 @@ import path from 'node:path';
// first; it proved unreliable (some workers were torn down before running
// it), which is why this is a single run-level hook instead.
//
// Rooted at /tmp rather than nested inside the current os.tmpdir(): macOS's
// per-user TMPDIR (/var/folders/.../T/) is already close to the 104-byte
// sun_path limit AF_UNIX sockets need, and tests that bind real sockets
// (e.g. runner-usbmux.test.ts) started hitting EINVAL once nested one level
// deeper. /tmp is short enough to leave headroom for those.
//
// check-tmpdir-leaks.ts imports these two rather than recomputing them, so
// the two can't drift onto different directories (os.tmpdir() != /tmp on
// macOS, where TMPDIR is a deep per-user path).
export const TEST_RUN_TMP_ROOT = '/tmp';
export const TEST_RUN_TMP_PREFIX = 'agent-device-test-run-';
// The root/prefix live in check-tmpdir-leaks-model.ts, next to the liveness
// classification this setup and the post-run leak check both rely on.
let testRunTmpDir: string;
let previousTmpDir: string | undefined;
@@ -45,6 +42,7 @@ export function setup(): void {
'agent-device-swift-cache',
);
}
reportPrunedRunDirectories(pruneAbandonedRunDirectories(TEST_RUN_TMP_ROOT));
// The pid is embedded so check-tmpdir-leaks.ts can tell a directory that's
// still in active use (its vitest process is alive — a concurrent run in
// another worktree, say) apart from one actually abandoned by a killed