mirror of
https://github.com/vercel/next.js.git
synced 2026-09-20 02:25:18 +08:00
Turbopack: terminate failed plugin worker threads (#96592)
## What Terminate a worker-thread plugin runtime when an evaluation or IPC error marks its `WorkerOperation` non-reusable. `disallow_reuse()` already removed the worker from pool statistics and disabled the callback that returns it to the pool, but it did not stop the underlying Node.js `Worker`. The worker then stayed strongly owned by `loaderWorkers`, kept its V8 isolate/module graph alive, and waited forever for another task that could never arrive. This calls the existing worker terminator used by `wait_or_kill`, scale-down, and scale-to-zero. It removes the routed native channel and dispatches the existing JavaScript `Worker.terminate()` callback, which also deletes the map entry. ## Why this matters for v0 v0's planned worker-thread runtime turns transient loader/PostCSS errors during agent edits into a deterministic lifecycle leak. On the real v0 `/button` route, each unique Tailwind/PostCSS failure retained one complete worker runtime on the baseline. The balanced eight-process A/B panel (80 unique failures per lane) measured: | Lane | `WorkerThread` slope | Private-memory slope | Median failure | Median recovery | >5 s failures | |---|---:|---:|---:|---:|---:| | baseline | **+1.000/error** | **+49.81 MiB/error** | 108.04 ms | 329.26 ms | 3 / 80 | | candidate | **0.000/error** | **+5.12 MiB/error** | 108.15 ms | 317.65 ms | 2 / 80 | That is a **100% elimination of leaked-worker growth** and an **89.7% reduction in retained private-memory slope** (exact process-bootstrap 95% interval: 85.7%–99.5%). Every candidate process was below every baseline process; exact 4-vs-4 process-label permutation `p=0.0143` one-sided / `0.0286` two-sided. Median failure latency changed by only +0.11 ms. The independent 100-cycle-per-lane endurance confirmation showed the same causal result: | Lane | `WorkerThread` cycle 1→100 | Private memory cycle 1→100 | Private slope | Median failure | Median recovery | |---|---:|---:|---:|---:|---:| | baseline | **38→137** | **4.28→7.34 GiB** | **+27.02 MiB/error** | 104.45 ms | 331.32 ms | | candidate | **32→32** | **3.93→4.10 GiB** | **+0.98 MiB/error** | 102.94 ms | 297.24 ms | That long run eliminated all observed worker growth and reduced the fitted private-memory slope 96.4%. Every one of the 200 unique failures recovered to the complete real route, and the source hash was restored after every cycle. The benchmark used fresh `.next` state per process, five error/recovery warmups, a five-minute quiescence window, unique uncached errors, exact source restoration after every cycle, and within-process slopes. Baseline and candidate native binaries were built from the same Next.js base, with only this lifecycle change affecting candidate native code, and selected by SHA-verified `NEXT_TEST_NATIVE_DIR` paths. This is independent of #96433. That PR coordinates explicit multi-file edit transactions; this PR fixes the lifecycle of a worker that has already failed. It also does not change idle-worker scale-down policy: failed workers are absent from the idle pool, so scale-down cannot see them. ## Regression test The new development test schedules a one-second filesystem marker inside a custom loader, then throws an evaluation error under `turbopackPluginRuntimeStrategy: 'workerThreads'`: - baseline native binary: fails because the orphan worker remains alive and writes the marker; - candidate native binary: passes because the worker is actually terminated; - candidate then renders a recovered value, proving replacement/recovery works. ## Validation - `pnpm build-all` - `cargo fmt --all -- --check` - `cargo check -p turbopack-node --all-targets` - `cargo test -p turbopack-node --lib` - `cargo clippy -p turbopack-node --all-targets -- -D warnings` - focused Prettier and ESLint checks for the new test - candidate integration test pass; exact baseline integration test fails at the intended liveness assertion - GPT-5.6 Sol xhigh + Claude Opus 5 xhigh autoreview panel: zero actionable findings, patch correct at 0.96 confidence Exact optimized native SHA-256 values used by the benchmark: - baseline: `609eeeed41e1425f06136b54b7997e5b1b1add992169b17a797889dfb830dd7f` - candidate: `141211112ae47b8bd814aaab437060114e92f60a1bf6f23e1b0e153b1e4837cc` Both stripped binaries were 205,863,784 bytes. Raw per-process JSON, `/proc` samples, server logs, harness source, analysis output, and A/B plus endurance graphs are retained in the investigation archive. Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
This commit is contained in:
@@ -0,0 +1,8 @@
|
||||
import { ReactNode } from 'react'
|
||||
export default function Root({ children }: { children: ReactNode }) {
|
||||
return (
|
||||
<html>
|
||||
<body>{children}</body>
|
||||
</html>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
const data = require('../input.probe')
|
||||
|
||||
export default function Page() {
|
||||
return <p>{data.default}</p>
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
const fs = require('node:fs')
|
||||
|
||||
module.exports = function errorLoader(source) {
|
||||
const value = source.trim()
|
||||
if (value === 'throw') {
|
||||
const { marker } = this.getOptions()
|
||||
setTimeout(() => fs.writeFileSync(marker, 'worker survived'), 1000)
|
||||
throw new Error('EXPECTED_WORKER_THREAD_LOADER_ERROR')
|
||||
}
|
||||
|
||||
return `export default ${JSON.stringify(value)}`
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
initial
|
||||
@@ -0,0 +1,23 @@
|
||||
const path = require('node:path')
|
||||
|
||||
/** @type {import('next').NextConfig} */
|
||||
module.exports = {
|
||||
experimental: {
|
||||
turbopackPluginRuntimeStrategy: 'workerThreads',
|
||||
},
|
||||
turbopack: {
|
||||
rules: {
|
||||
'*.probe': {
|
||||
as: '*.js',
|
||||
loaders: [
|
||||
{
|
||||
loader: require.resolve('./error-loader.js'),
|
||||
options: {
|
||||
marker: path.join(__dirname, 'worker-survived.txt'),
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
+34
@@ -0,0 +1,34 @@
|
||||
import { nextTestSetup } from 'e2e-utils'
|
||||
import { retry, waitFor } from 'next-test-utils'
|
||||
|
||||
describe('turbopack worker thread error cleanup', () => {
|
||||
const { next, isTurbopack } = nextTestSetup({
|
||||
files: __dirname,
|
||||
})
|
||||
|
||||
const itOnlyTurbopack = isTurbopack ? it : it.skip
|
||||
|
||||
itOnlyTurbopack(
|
||||
'terminates a loader worker after an evaluation error',
|
||||
async () => {
|
||||
expect((await next.render$('/'))('p').text()).toBe('initial')
|
||||
|
||||
await next.patchFile('input.probe', 'throw')
|
||||
await waitFor(1000)
|
||||
await next.fetch('/')
|
||||
await retry(async () => {
|
||||
expect(next.cliOutput).toContain('EXPECTED_WORKER_THREAD_LOADER_ERROR')
|
||||
}, 30_000)
|
||||
|
||||
// The failing loader schedules this marker before throwing. A worker that
|
||||
// is merely removed from the pool remains alive and writes it later.
|
||||
await waitFor(2000)
|
||||
expect(await next.hasFile('worker-survived.txt')).toBe(false)
|
||||
|
||||
await next.patchFile('input.probe', 'recovered')
|
||||
await retry(async () => {
|
||||
expect((await next.render$('/'))('p').text()).toBe('recovered')
|
||||
})
|
||||
}
|
||||
)
|
||||
})
|
||||
@@ -301,6 +301,8 @@ impl Operation for WorkerOperation {
|
||||
if self.on_drop.is_some() {
|
||||
self.state.stats.lock().remove_worker();
|
||||
self.on_drop = None;
|
||||
// Clearing the return-to-pool callback does not stop the underlying Node.js worker.
|
||||
let _ = terminate_worker(self.worker_options.clone(), self.worker_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user