Turbopack: terminate failed plugin worker threads (#96592)

## What

Terminate a worker-thread plugin runtime when an evaluation or IPC error
marks
its `WorkerOperation` non-reusable.

`disallow_reuse()` already removed the worker from pool statistics and
disabled
the callback that returns it to the pool, but it did not stop the
underlying
Node.js `Worker`. The worker then stayed strongly owned by
`loaderWorkers`, kept
its V8 isolate/module graph alive, and waited forever for another task
that
could never arrive.

This calls the existing worker terminator used by `wait_or_kill`,
scale-down,
and scale-to-zero. It removes the routed native channel and dispatches
the
existing JavaScript `Worker.terminate()` callback, which also deletes
the map
entry.

## Why this matters for v0

v0's planned worker-thread runtime turns transient loader/PostCSS errors
during
agent edits into a deterministic lifecycle leak. On the real v0
`/button`
route, each unique Tailwind/PostCSS failure retained one complete worker
runtime
on the baseline.

The balanced eight-process A/B panel (80 unique failures per lane)
measured:

| Lane | `WorkerThread` slope | Private-memory slope | Median failure |
Median recovery | >5 s failures |
|---|---:|---:|---:|---:|---:|
| baseline | **+1.000/error** | **+49.81 MiB/error** | 108.04 ms |
329.26 ms | 3 / 80 |
| candidate | **0.000/error** | **+5.12 MiB/error** | 108.15 ms | 317.65
ms | 2 / 80 |

That is a **100% elimination of leaked-worker growth** and an **89.7%
reduction
in retained private-memory slope** (exact process-bootstrap 95%
interval:
85.7%–99.5%). Every candidate process was below every baseline process;
exact
4-vs-4 process-label permutation `p=0.0143` one-sided / `0.0286`
two-sided.
Median failure latency changed by only +0.11 ms.

The independent 100-cycle-per-lane endurance confirmation showed the
same
causal result:

| Lane | `WorkerThread` cycle 1→100 | Private memory cycle 1→100 |
Private slope | Median failure | Median recovery |
|---|---:|---:|---:|---:|---:|
| baseline | **38→137** | **4.28→7.34 GiB** | **+27.02 MiB/error** |
104.45 ms | 331.32 ms |
| candidate | **32→32** | **3.93→4.10 GiB** | **+0.98 MiB/error** |
102.94 ms | 297.24 ms |

That long run eliminated all observed worker growth and reduced the
fitted
private-memory slope 96.4%. Every one of the 200 unique failures
recovered to
the complete real route, and the source hash was restored after every
cycle.

The benchmark used fresh `.next` state per process, five error/recovery
warmups,
a five-minute quiescence window, unique uncached errors, exact source
restoration
after every cycle, and within-process slopes. Baseline and candidate
native
binaries were built from the same Next.js base, with only this lifecycle
change
affecting candidate native code, and selected by SHA-verified
`NEXT_TEST_NATIVE_DIR` paths.

This is independent of #96433. That PR coordinates explicit multi-file
edit
transactions; this PR fixes the lifecycle of a worker that has already
failed.
It also does not change idle-worker scale-down policy: failed workers
are absent
from the idle pool, so scale-down cannot see them.

## Regression test

The new development test schedules a one-second filesystem marker inside
a
custom loader, then throws an evaluation error under
`turbopackPluginRuntimeStrategy: 'workerThreads'`:

- baseline native binary: fails because the orphan worker remains alive
and
  writes the marker;
- candidate native binary: passes because the worker is actually
terminated;
- candidate then renders a recovered value, proving replacement/recovery
works.

## Validation

- `pnpm build-all`
- `cargo fmt --all -- --check`
- `cargo check -p turbopack-node --all-targets`
- `cargo test -p turbopack-node --lib`
- `cargo clippy -p turbopack-node --all-targets -- -D warnings`
- focused Prettier and ESLint checks for the new test
- candidate integration test pass; exact baseline integration test fails
at the
  intended liveness assertion
- GPT-5.6 Sol xhigh + Claude Opus 5 xhigh autoreview panel: zero
actionable
  findings, patch correct at 0.96 confidence

Exact optimized native SHA-256 values used by the benchmark:

- baseline:
`609eeeed41e1425f06136b54b7997e5b1b1add992169b17a797889dfb830dd7f`
- candidate:
`141211112ae47b8bd814aaab437060114e92f60a1bf6f23e1b0e153b1e4837cc`

Both stripped binaries were 205,863,784 bytes. Raw per-process JSON,
`/proc`
samples, server logs, harness source, analysis output, and A/B plus
endurance
graphs are retained in the investigation archive.

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
This commit is contained in:
Marcos Hernanz
2026-08-04 13:29:48 -07:00
committed by GitHub
parent 55366c116d
commit 39b7da2ee8
7 changed files with 85 additions and 0 deletions
@@ -0,0 +1,8 @@
import { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
@@ -0,0 +1,5 @@
const data = require('../input.probe')
export default function Page() {
return <p>{data.default}</p>
}
@@ -0,0 +1,12 @@
const fs = require('node:fs')
module.exports = function errorLoader(source) {
const value = source.trim()
if (value === 'throw') {
const { marker } = this.getOptions()
setTimeout(() => fs.writeFileSync(marker, 'worker survived'), 1000)
throw new Error('EXPECTED_WORKER_THREAD_LOADER_ERROR')
}
return `export default ${JSON.stringify(value)}`
}
@@ -0,0 +1,23 @@
const path = require('node:path')
/** @type {import('next').NextConfig} */
module.exports = {
experimental: {
turbopackPluginRuntimeStrategy: 'workerThreads',
},
turbopack: {
rules: {
'*.probe': {
as: '*.js',
loaders: [
{
loader: require.resolve('./error-loader.js'),
options: {
marker: path.join(__dirname, 'worker-survived.txt'),
},
},
],
},
},
},
}
@@ -0,0 +1,34 @@
import { nextTestSetup } from 'e2e-utils'
import { retry, waitFor } from 'next-test-utils'
describe('turbopack worker thread error cleanup', () => {
const { next, isTurbopack } = nextTestSetup({
files: __dirname,
})
const itOnlyTurbopack = isTurbopack ? it : it.skip
itOnlyTurbopack(
'terminates a loader worker after an evaluation error',
async () => {
expect((await next.render$('/'))('p').text()).toBe('initial')
await next.patchFile('input.probe', 'throw')
await waitFor(1000)
await next.fetch('/')
await retry(async () => {
expect(next.cliOutput).toContain('EXPECTED_WORKER_THREAD_LOADER_ERROR')
}, 30_000)
// The failing loader schedules this marker before throwing. A worker that
// is merely removed from the pool remains alive and writes it later.
await waitFor(2000)
expect(await next.hasFile('worker-survived.txt')).toBe(false)
await next.patchFile('input.probe', 'recovered')
await retry(async () => {
expect((await next.render$('/'))('p').text()).toBe('recovered')
})
}
)
})
@@ -301,6 +301,8 @@ impl Operation for WorkerOperation {
if self.on_drop.is_some() {
self.state.stats.lock().remove_worker();
self.on_drop = None;
// Clearing the return-to-pool callback does not stop the underlying Node.js worker.
let _ = terminate_worker(self.worker_options.clone(), self.worker_id);
}
}
}