Files
civitai__civitai/vitest.config.mts
T
Zachary Lowden a4e886fc9d test(get-models-raw): hoist model.service import out of the per-test timeout budget (#3612)
* test(get-models-raw): hoist model.service import out of the per-test timeout budget

`get-models-raw.transient-503.test.ts` loaded the module under test with
`await import('~/server/services/model.service')` inside a test helper, so the
first test to run paid the cold TS transform + import of a ~4,800-line module
and its transitive service graph — and Vitest charges that to that test's
`testTimeout`. Measured, the file spent ~99.9% of its runtime in one test, and
that share is invariant to CPU (99.86% unconstrained on a 24-core host, 99.85%
under a 4-CPU quota). Against the 60s ceiling that left the file passing or
failing on ambient CI-runner speed, presenting as a single
`Test timed out in 60000ms` rather than a uniformly slow file.

Hoist the import to module scope. `vi.mock` is hoisted above static imports, so
the mocks still apply and no test semantics change; the transform now happens in
Vitest's collection phase, which no timeout bounds (Vitest exposes only
testTimeout / hookTimeout / teardownTimeout). This removes the cliff rather than
moving it, which raising the timeout or deferring to `beforeAll` would do.

Worst per-test, measured at three points (before -> after):
  isolated, unconstrained 24-core host   4364ms -> 2ms
  isolated, 4-CPU quota                  3685ms -> 4ms
  full 783-file suite, 4-CPU quota       2726ms -> 2ms

Suite unchanged: 783 files, 11588 passed + 1 skipped, before and after.

Mutation-verified that the 9 tests still discriminate (production code restored
afterwards; `git diff` on model.service.ts empty):
  isTransientMeiliError(err) -> err instanceof MeiliCallTimeoutError
    => 6 failed / 3 passed, exactly the widening cases, each failing because the
       raw MeiliSearchCommunicationError escaped unconverted
  isTransientMeiliError(err) -> true
    => 2 failed / 7 passed, exactly the `does NOT convert` negatives, each
       failing because a real app bug was masked as a retryable 503

Also adds a comment beside `testTimeout` in vitest.config.mts: the existing note
rationalised the 60s ceiling as the thing that absorbs cold `await import()`
cost, which normalises the pattern that caused this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(test): mark the CI-runner figure as a PROJECTION, not a measurement

Comment-only. The hoist comment presented two numbers under one "measured:"
label, but only the first was measured:

  measured   2726ms of a 2730ms file, locally under a 4-CPU quota — and the
             ~99.9% first-test share is invariant to CPU (99.86% unconstrained
             vs 99.85% under quota)
  PROJECTED  ~44.2s of the observed ~44.3s file total on the CI runner. Nobody
             reproduced this in-pod; it is that invariant share applied to a
             file total read out of a CI log.

Labelling a projection as a measurement is how a plausible number gets cited
later as a result. It is stated as a projection now, with the two things that
corroborate it: the failure signature was ONE test timing out (not a uniformly
slow file), and the same file — byte-identical — passed at 45.3s on a faster
runner and timed out at 60.2s on a slower one.

The projection being unverified does not weaken the fix. The mechanism is
directly observable locally: after the hoist this file reports
`9 tests | 7ms` with `transform 3.44s, import 5.78s` — the import cost is real
and now lands in Vitest's COLLECTION phase, which no timeout bounds, instead of
inside one test's 60s budget.

9/9 pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 08:37:37 -05:00

176 lines
9.5 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { defineConfig } from 'vitest/config';
import { playwright } from '@vitest/browser-playwright';
import path from 'path';
// Mirror the workspace `@civitai/*` package mappings from tsconfig.json `paths` so Vitest
// (which doesn't read tsconfig paths, and these packages aren't symlinked into the root
// node_modules) resolves them the same way the app build does. `@civitai/auth` is omitted —
// it IS symlinked in node_modules and resolves via its own exports map. Subpath regex must come
// before the bare entry so `@civitai/redis/client` doesn't get caught by the bare `@civitai/redis`.
const civitaiWorkspacePkgs = [
'db-schema',
'db',
'redis',
'clickhouse',
'axiom',
'telemetry',
'brand',
// `@civitai/notifications` (packages/civitai-notifications) is re-exported by
// src/server/common/enums.ts; without this alias Vitest can't resolve it and
// the whole server suite cascades (enums.ts → BlockRegistry undefined → …).
'notifications',
// `@civitai/buzz` (packages/civitai-buzz) is imported by
// src/shared/constants/buzz.constants.ts + src/server/services/buzz.service.ts —
// both pulled in transitively by blocks.router.ts and model-version.service.ts.
// Same story as notifications: not symlinked into root node_modules, so without
// this alias every suite touching the buzz chain fails to collect.
'buzz',
];
const civitaiAlias = civitaiWorkspacePkgs.flatMap((p) => {
const src = path.resolve(__dirname, `packages/civitai-${p}/src`).replace(/\\/g, '/');
return [
{ find: new RegExp(`^@civitai/${p}/(.*)$`), replacement: `${src}/$1` },
{ find: `@civitai/${p}`, replacement: `${src}/index` },
];
});
const alias = [{ find: '~', replacement: path.resolve(__dirname, './src') }, ...civitaiAlias];
// Browser-mode (`component` project) alias: stub the native `sharp` module.
// A few `.browser.test.tsx` tests import a Next *page* to render its client shell;
// the page's `getServerSideProps` transitively pulls a server service that does
// `import sharp from 'sharp'`. Next strips that server-only graph from real client
// builds, but Vitest's browser build does not — so esbuild's optimizeDeps scan
// follows the import into sharp and dies bundling its native
// `require('../build/Release/sharp-*.node')`, killing the WHOLE component suite
// before any test runs. (The tests `vi.mock` server-side-helpers, but that is a
// runtime interception and can't stop the build-time static scan.) The `unit`
// (node) project keeps the real sharp. Must precede the `~` entry so it wins.
const componentAlias = [
{ find: /^sharp$/, replacement: path.resolve(__dirname, 'test/stubs/sharp.ts') },
...alias,
];
// Two Vitest projects sharing one config/runner:
// - `unit` = the existing node-env suite, unchanged.
// - `component` = browser-mode (real Chromium via Playwright) for React
// components/widgets. Distinct `.browser.test.tsx` glob so the
// unit project never boots a browser (its include is `.test.ts`
// only, so `.tsx` is already excluded — the glob is explicit
// belt-and-suspenders). See `test/component-setup.tsx`.
export default defineConfig({
resolve: { alias },
test: {
projects: [
{
resolve: { alias },
test: {
name: 'unit',
globals: true,
environment: 'node',
include: ['src/**/*.test.ts'],
exclude: ['node_modules', 'tests/**/*'], // Exclude Playwright tests
setupFiles: ['src/__tests__/setup.ts'],
// Several unit tests cold-`await import(...)` a large Next API-page / service
// module graph (mocked I/O, but a real ~916s TS transform). With the suite's
// worker pool saturated, that legitimate cold transform races for CPU and
// overran the old 10s default — a PASS→FAIL that tracked CI load, not code.
// 60s absorbs that contention while still bounding a genuine hang (these are
// mocked-I/O tests; nothing should legitimately approach a minute).
//
// 🔴 But do NOT treat this ceiling as the place to solve that cost. A cold
// `await import(...)` reached from a TEST BODY is charged to that ONE test's
// budget, so the file's whole transform lands inside a single 60s clock:
// get-models-raw.transient-503 spent 99.9% of its runtime in one test that
// way (2726ms of a 2730ms file under a 4-CPU quota) and went red purely on
// ambient runner speed. Hoist the import to MODULE SCOPE instead — `vi.mock`
// is hoisted above imports, so the mocks still apply, and the transform then
// lands in Vitest's COLLECTION phase, which no timeout bounds (Vitest has
// only testTimeout / hookTimeout / teardownTimeout). Where a module genuinely
// must load per-suite, a `beforeAll` with an explicit long timeout is the
// fallback (see prisma-inconsistent-orphan-relations). Raising this number
// only moves the cliff.
testTimeout: 60000,
// Same cold-`await import()` graph is paid in some suites' beforeAll/beforeEach
// (e.g. file-download-lookup, listForModel.behavior). Vitest's default
// hookTimeout is 10s — too tight for that transform on a saturated CI box — so
// match testTimeout. Without this a hoisted import flakes the hook instead.
hookTimeout: 60000,
deps: {
inline: [/@civitai\/client/],
},
},
},
{
// dedupe React so a transitive dep can't pull a second copy — a second
// React makes `useContext` read a null context and crashes some Mantine
// components (e.g. @mantine/dropzone) in browser mode, notably on a COLD
// optimizeDeps cache (fresh CI runs). Canonical fix; protects every
// component test from this class of dual-React crash.
resolve: { alias: componentAlias, dedupe: ['react', 'react-dom'] },
// Pre-bundle deps the component setup mocks/imports so Vitest doesn't
// discover them mid-run and trigger a "Vite unexpectedly reloaded a
// test" warning (a flake vector).
//
// On a COLD optimizeDeps cache (every fresh CI/preview run), Vite was
// discovering `vitest-browser-react` + `react/jsx-dev-runtime` only when
// `test/component-setup.tsx` first imported them — mid-run — and reloading.
// The reload tears down the module context while component-setup is still
// importing `vitest-browser-react`, so that package evaluates OUTSIDE a live
// vitest runner → "Vitest failed to find the runner" → "Failed to import test
// file test/component-setup.tsx" → EVERY component test fails to load. (Warm
// cache hid it: the 2nd local run always passed.) Pre-bundling them here makes
// the optimize pass happen BEFORE the run starts, so there's no mid-run reload.
optimizeDeps: {
include: ['next/router', 'vitest-browser-react', 'react/jsx-dev-runtime', 'react/jsx-runtime'],
// `@vitest/browser` seeds optimizeDeps.entries from EVERY `*.browser.test.tsx` file
// (globTestFiles), not just the one you ran. The review app-listing browser tests
// (src/tests/pages/apps/review/review-{detail-page,queue-nav}.browser.test.tsx, from
// #3298 on main) each import their page, which pulls in `createServerSideProps` ->
// `appRouter` -> app-listing-assets.service.ts's
// `await import('sharp')`. Those tests `vi.mock` `server-side-helpers` so `sharp` is
// never evaluated at runtime — but esbuild's static scan follows the import anyway and
// can't pre-bundle sharp's native binding (a template-literal `require` of a `.node`
// file), failing the whole component project with "No loader is configured for '.node'
// files" regardless of which test you targeted. Exclude it; no component test needs it.
exclude: ['sharp'],
},
test: {
name: 'component',
globals: true,
include: ['src/**/*.browser.test.tsx'],
// process-shim MUST come first (no imports) so it runs before any
// component's module graph reads `process.env` at import time.
setupFiles: ['test/browser-process-shim.ts', 'test/component-setup.tsx'],
browser: {
enabled: true,
// `--no-sandbox` + `--disable-dev-shm-usage`: required to launch
// Chromium as root inside the Tekton CI container (node:20 pod runs
// as UID 0; without --no-sandbox Chromium refuses to start, and the
// container's small /dev/shm crashes it without --disable-dev-shm-usage).
// Harmless locally.
// CI uses Playwright's bundled Chromium (env unset). NixOS can't run
// that generic binary; point this at a system Chromium, e.g.
// `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH=$(command -v chromium)`.
provider: playwright({
launchOptions: {
args: ['--no-sandbox', '--disable-dev-shm-usage'],
...(process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH
? { executablePath: process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH }
: {}),
},
}),
headless: true,
instances: [{ browser: 'chromium' }],
},
},
},
],
coverage: {
provider: 'v8',
reporter: ['text', 'html'],
include: ['src/server/services/**', 'src/server/jobs/**'],
},
},
});