Files
civitai__civitai/tests/preview-apps-marketplace.spec.ts
T
Zachary Lowden 7ebad80173 test(ci): fix failing unit + component + smoke suites on main (#2762)
* test(ci): fix failing unit + smoke suites (flaky perf budgets, cold-import timeouts, stale heading)

Three CI suites were red on recent PRs. Root-caused and fixed each
(no skips, no disabled tests, no loosened correctness assertions).

Unit suite (`pnpm test:unit:run`):
- ReDoS perf guards (audit-redos / audit-cjk-redos / audit-gate-perf)
  asserted absolute wall-clock budgets (`expect(ms).toBeLessThan(100)`)
  on a single regex call over a long input. That measures CPU speed, not
  code: the same (provably LINEAR) code ran ~15ms on a fast runner and
  ~300-560ms on a loaded one, flaking PASS->FAIL purely on hardware. The
  real invariant is asymptotic — the `{0,200}` gap bound + zero-width word
  boundaries made the audit O(n), not the pre-fix O(n^2)/exponential
  (11-84s of synchronous CPU = user-triggerable DoS). Replaced the
  hardware-coupled budgets with `expectSubQuadraticScaling` (new shared
  `redos-perf-helpers.ts`): a large-N absolute ceiling with an
  order-of-magnitude margin over linear but far below the known quadratic
  cost, plus a small->large scaling-ratio check (skipped below the timer
  noise floor). Hardware-independent, and STRICTER against a reintroduced
  ReDoS than the old budgets. All correctness assertions kept verbatim.
  (Verified linear empirically: includesMinor 8.6/16/19/47/96ms at
  N=2k/4k/8k/16k/32k.)
- Several tests cold-`await import(...)` a large Next API-page / service
  module graph (mocked I/O, but a real ~9-16s TS transform). Under a
  saturated worker pool that legitimate transform raced for CPU and
  overran the 10s global timeout, cascading the next test into a
  half-loaded-module assertion. Hoisted the heavy import into `beforeAll`
  (paid once, off the per-test budget) for model.service +
  /api/v1/models, gave the single-test catalog-cors-wiring an explicit
  timeout, and raised the global unit `testTimeout` 10s->60s (these are
  mocked-I/O tests; nothing should legitimately approach a minute).

Smoke suite (`tests/preview-apps-marketplace.spec.ts:90`):
- Asserted a `getByRole('heading', { name: 'Civitai App Blocks' })` that
  the app-blocks nav refactor (#2749/#2758) DELIBERATELY removed —
  `AppsPageLayout` now omits the title on the marketplace surface. Updated
  the stale assertion to the always-on `AppsSubNav` "Marketplace" tab
  (`getByRole('tab', { name: 'Marketplace' })`), which uniquely identifies
  the rendered apps surface (proves the appBlocks SSR gate cleared, not a
  404). Selector validated against AppsSubNav.tsx and the passing
  AppsSubNav component test's real DOM (`<a href role="tab">`).

Component suite (`pnpm test:component`): green on this tree (26 files /
264 tests) against the current source incl. the nav refactor — no
component test references the removed heading; no change needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(component): fix CliSubmitCta copy tests failing in CI headless Chromium

The component suite failed on recent PRs (preview / component-tests) on
src/components/Apps/CliSubmitCta.browser.test.tsx — the two copy tests that
assert the "Copied" affordance renders after activating the copy button.

Root cause: Mantine's `useClipboard` calls `navigator.clipboard.writeText`.
In CI's headless Chromium the page is an insecure context with no clipboard
permission, so the real `writeText` rejects — `copied` never flips and "Copied"
never renders. The tests passed locally only because a desktop Chromium grants
the permission. Reproduced the exact CI failure by forcing `writeText` to reject
(same "Cannot find element: getByText('Copied')" error).

Fix: stub a resolving `navigator.clipboard` in the shared browser-mode setup
(test/component-setup.tsx) so copy behaviour is deterministic and matches a real
secure-context browser, independent of the CI Chromium's permission state. The
tests assert the "Copied" UI state, not OS clipboard contents. Verified: copy
tests now pass (8/8, 751ms vs the prior 2x15s timeouts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(slow-log): drain emit tail deterministically (fix CI flake on loaded box)

CI unit suite failed on 2 tests: trpc-slow-log.test.ts:66 and
audit-slow-log.test.ts:74 — both `expect(logToAxiom).toHaveBeenCalledTimes(1)`
got 0. They passed locally but fail on the saturated CI box.

Root cause: emitSlowLog in both modules is a fire-and-forget async tail that
`await import('~/server/logging/client')` (audit also awaits node:crypto) before
calling logToAxiom. The tests drained it with a FIXED ~25ms wall-clock flush
(5x setTimeout(5ms)). On a loaded CI runner (this run: transform 467s / import
1189s) the dynamic import resolves slower than 25ms, so logToAxiom hasn't been
called when the assertion runs → 0 calls. Pure timeout-vs-load race.

Fix: track the in-flight emit promise in a module-level Set and expose a
`__flushPendingEmitsForTest()` hook (consistent with the existing __reset* /
__rateGate* test hooks). The tests' `flush()` now awaits the actual emit instead
of a fixed timeout — deterministic regardless of CI load. Production behaviour is
unchanged: still fire-and-forget; the Set entry is removed on settle (no
retention). Verified: 29/29 pass, trpc file in 24ms.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(redos): stop the scaling-ratio guard flaking on a contended CI box

The unit suite still had 1 failure on CI: audit-redos.test.ts:182 —
`includesMinor "young"+run: cost scaled 11.99x when input grew 4x` (ratio
ceiling 8x). The op is genuinely linear (9.3ms→111.7ms) and the PRIMARY absolute
guard passed (111ms ≪ 1500ms); only the SECONDARY scaling-ratio check tripped.

Root cause: at factor=4 the ratio ceiling (8x) sits between linear (4x) and
quadratic (16x) with too little margin, and the baseline (9.3ms) was barely above
the 8ms noise floor. On a shared/contended CI core, contention adds variable
absolute ms to each short measurement independently, so a ~10ms baseline against a
contention-inflated large reads as a false 12x — pure measurement noise, not a
ReDoS.

Fix (redos-perf-helpers.ts):
- Raise NOISE_FLOOR_MS 8→40 so the ratio is skipped when the baseline is too
  small to be stable. The absolute LARGE_N (1500ms) + HANG (5s) ceilings carry
  correctness in that regime (the documented design).
- Widen the ratio ceiling: quadraticGuard 2→3.5 (ceiling 8→14), giving linear
  work 3.5x slack over its expected 4x while still tripping a true O(n²) (≥16x).

The absolute large-N guard remains the load-bearing ReDoS check (linear ~100ms,
removed O(n²) ~2800ms+); the ratio is now a secondary clearly-egregious backstop.
Verified: all 3 ReDoS suites green (27/27).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(unit): kill remaining CI-contention flakes (redos absolute ceiling, cold-import hook timeouts)

The unit suite on the preview CI box (this run: transform 467s / import 1189s —
extreme worker-pool contention) surfaced two more load-sensitive failures:

1) audit-redos.test.ts — the ratio fix worked (ratio read ~4x = linear), but the
   PRIMARY absolute guard then tripped: genuinely-linear work measured 1900ms,
   over the 1500ms LARGE_N_LINEAR_CEILING. A single absolute wall-clock ceiling
   can't be both tight-enough-to-catch-a-mild-quadratic and loose-enough-to-never-
   flake on this box. Fix (redos-perf-helpers.ts): split into two regimes —
   - reliable-ratio regime (both medians ≥ 40ms noise floor, i.e. a slow/contended
     box): the SHAPE ratio is authoritative (hardware-independent) + only the
     generous 5s HANG ceiling as an absolute backstop.
   - fast-box regime (baseline below the floor, ratio noise-dominated): the tight
     1500ms ceiling, which has enormous margin on a fast box.
   Both regimes catch a reintroduced O(n²); neither false-trips on linear.

2) file-download-lookup.test.ts — `Test timed out in 30000ms`. The describe had a
   `{ timeout: 30000 }` override (below the 60s global) and each test did a lazy
   `await import('../file.service')` — a heavy cold module graph that exceeds 30s
   under contention. Fix: hoist the import into beforeAll (paid once, 60s hook
   timeout), drop the tight override, tests use the warm reference.

3) vitest.config.mts — add `hookTimeout: 60000` to the unit project. testTimeout
   was already 60s but hookTimeout defaulted to 10s, so any beforeAll/beforeEach
   cold-import (file-download-lookup, listForModel.behavior, ...) would flake the
   HOOK instead of the test. One-line global hedge for the whole class.

Verified locally: redos suites + file-download-lookup green (31/31).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(component): raise vi.waitFor default timeout to absorb CI contention

PageBlockHost.browser.test.tsx flaked on the preview CI box: "after BLOCK_READY,
REQUEST_CONSENT opens the consent dialog" — `expected [] to have length 1`. The
test already wraps the assertion in vi.waitFor (correct), but vi.waitFor defaults
to a 1000ms timeout, and on the saturated box (browser tests share the host with
the image build) the postMessage→consent-dialog round-trip exceeded 1s, so the
waitFor expired → empty dialog set → PASS→FAIL on load, not code.

The browser suite has ~80 vi.waitFor sites and NONE pass an explicit timeout, so
any of them awaiting an async round-trip (postMessage, tRPC settle, zustand
update) is a latent 1000ms-vs-contention flake. Rather than edit every call site,
raise the DEFAULT globally in the shared browser setup (test/component-setup.tsx):
wrap vi.waitFor so a call that omits `timeout` gets 10s; calls that pass their own
timeout (e.g. the 10s/15s terminal-path tests) are untouched. One root-cause fix
for the whole class. Verified: PageBlockHost + CliSubmitCta green (20/20).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:38:35 -05:00

177 lines
9.0 KiB
TypeScript

import { expect, test } from '@playwright/test';
import { storageStatePath } from './preview-fixtures';
import { trpcQuery } from './preview-trpc';
/**
* Preview-e2e (F-C): App Blocks MARKETPLACE discovery + per-app detail — the
* anon-capable PUBLIC read path (`blocks.listAvailable` → `blocks.getAppDetail`)
* plus the two host pages that render them (`/apps`, `/apps/<appBlockId>`).
* Otherwise untested by the preview suite; a PR that broke the marketplace
* listing/detail projection or the `features.appBlocks` page gate passes every
* other preview spec today.
*
* Runs as the `mod` fixture (id 2000000001) — the ONLY preview role with
* `features.appBlocks` (the Flipt `app-blocks-enabled` flag is moderator-segment
* -only) AND exempt from the per-IP marketplace rate limit (listAvailable /
* getAppDetail carry a 60/60 rateLimit that the middleware waives for mods).
* The non-mod testers do NOT have appBlocks: for them /apps SSR-resolves to a
* Next 404 (resolveAppsPageAccess → notFound) and listAvailable returns empty —
* so this MUST run as mod to exercise the real read path.
*
* Data resilience: the dev DB is a weekly prod clone — there are ~3 approved app
* blocks today, but a given clone could have zero. So we DISCOVER an appBlockId
* at runtime from `listAvailable` (never hardcode one) and `test.skip()` with an
* annotation when the clone has no approved blocks, rather than hard-failing.
*
* NB: these procs are DB-backed (not meili-backed search), so no `retryFlaky`.
*
* Verified tRPC shapes (against origin/main, paths relative to civitai/src):
* - blocks.listAvailable (blocks.router.ts:952 publicProcedure + enforceApp
* BlocksFlag + 60/60 rateLimit; input listAvailableSchema, all fields
* optional/defaulted so `{}` is valid) RETURNS AN OBJECT, NOT a bare array:
* `{ items: AvailableBlock[]; nextCursor?: string }`
* (BlockRegistry.listAvailable, block-registry.service.ts:2226). Each
* AvailableBlock (subscription.schema.ts:157) = { id, blockId, appId,
* appName, manifest: PublicBlockManifest, installCount, category,
* scopesSummary }. We DISCOVER an id from `items[0].id` (that id is the
* `appBlockId`).
* - blocks.getAppDetail (blocks.router.ts:1001 publicProcedure + flag +
* 60/60 rateLimit; input { appBlockId }) returns `PublicAppDetail | null`
* (subscription.schema.ts:371): { id, blockId, appId, appName, manifest:
* { name?, description?, targets? }, scopes: string[], contentRating,
* version, installCount, liveUrl, screenshots }. Returns null for a missing
* / non-approved id (the router then throws NOT_FOUND — our helper would
* surface that as a thrown error). We assert the shape of a discovered,
* known-approved id, so a non-null detail is expected.
* - Detail page (`/apps/[appBlockId]/index.tsx`) renders the block name as
* `<Title order={2}>{name}</Title>` where
* `name = detail.manifest.name ?? detail.blockId ?? appBlockId` — so the
* visible host-rendered name == `manifest.name || blockId`.
* - Marketplace page (`/apps/index.tsx`) renders the `AppsSubNav` tabs bar — its
* first tab is `{ href: '/apps', label: 'Marketplace' }` (AppsSubNav.tsx:55) —
* plus a search `TextInput` (placeholder "Search by name or block id"). It no
* longer renders a `<Title>Civitai App Blocks</Title>`: the app-blocks nav
* refactor (#2749/#2758) made `AppsPageLayout` DELIBERATELY OMIT the page title
* on the marketplace surface ("omit for a header with just the tabs, e.g. the
* marketplace" — AppsPageLayout.tsx:36), so the heading no longer exists by
* design. We assert the "Marketplace" tab instead — it uniquely identifies the
* rendered apps surface for an appBlocks-enabled viewer; a non-appBlocks viewer
* gets the Next 404 (resolveAppsPageAccess.ts → notFound).
*/
const ROLE = 'mod' as const;
// Public marketplace listing shape (the fields this spec reads). Mirrors
// AvailableBlock — typed locally so the tRPC result isn't `unknown`/implicit any.
type AvailableBlock = {
id: string;
blockId: string;
appId: string;
appName: string | null;
manifest: { name?: string; description?: string; targets?: Array<{ slotId?: string }> };
installCount: number;
category: string | null;
scopesSummary: string[];
};
type ListAvailableResult = { items: AvailableBlock[]; nextCursor?: string };
// Public per-app detail shape (the fields this spec reads). Mirrors
// PublicAppDetail — getAppDetail returns this or null.
type PublicAppDetail = {
id: string;
blockId: string;
appId: string;
appName: string | null;
manifest: { name?: string; description?: string; targets?: Array<{ slotId?: string }> };
scopes: string[];
contentRating: string | null;
version: string | null;
installCount: number;
liveUrl: string;
screenshots: Array<{ index: number; url: string; contentType: string }>;
};
test.describe('App Blocks marketplace discovery + detail render (mod)', () => {
test.use({ storageState: storageStatePath(ROLE) });
test('listAvailable → getAppDetail round-trip + /apps and /apps/[id] render', async ({
page,
}) => {
// The marketplace index renders for an appBlocks-enabled viewer (the mod) and
// 404s for everyone else. Asserting it loads (status < 400) + shows the
// marketplace's "Marketplace" sub-nav tab proves the mod cleared the
// `features.appBlocks` SSR gate and the page rendered (NOT the 404 a
// non-appBlocks user gets). The page no longer renders a "Civitai App Blocks"
// heading — the app-blocks nav refactor (#2749/#2758) made AppsPageLayout omit
// the title on the marketplace surface — so we assert the always-on
// "Marketplace" tab, which uniquely identifies the apps surface.
// domcontentloaded ONLY — never networkidle.
const resp = await page.goto('/apps', { waitUntil: 'domcontentloaded' });
expect(resp?.status(), 'GET /apps status for the appBlocks-enabled mod').toBeLessThan(400);
await expect(
page.getByRole('tab', { name: 'Marketplace' }),
'/apps should render the AppsSubNav "Marketplace" tab for an appBlocks-enabled mod (not a 404)'
).toBeVisible();
// DISCOVER an appBlockId at runtime from the public listing. Never hardcode
// one — the weekly dev clone's approved set varies. `{}` input is valid (all
// listAvailableSchema fields are optional/defaulted). page.request carries the
// mod cookie; the helper stamps Origin/Referer for the CSRF gate.
const listing = await trpcQuery<ListAvailableResult>(page.request, 'blocks.listAvailable', {});
expect(
Array.isArray(listing?.items),
'blocks.listAvailable should resolve to { items: AvailableBlock[] }'
).toBe(true);
const blocks = listing?.items ?? [];
test.skip(
blocks.length === 0,
'No approved app blocks in this dev-DB clone — nothing to discover (the weekly prod clone can have zero). Skipping the detail-render leg rather than hard-failing.'
);
// One representative block — don't crawl the whole list.
const first = blocks[0];
expect(typeof first.id, 'each listed block should carry a string id (the appBlockId)').toBe(
'string'
);
// Per-app DETAIL for that exact block. getAppDetail returns the public
// projection for an approved id; a discovered id IS approved (listAvailable
// only returns status='approved' rows), so detail must be non-null.
const detail = await trpcQuery<PublicAppDetail | null>(page.request, 'blocks.getAppDetail', {
appBlockId: first.id,
});
expect(
detail,
'getAppDetail should return a non-null detail for a discovered approved id'
).not.toBeNull();
expect(detail!.id, 'getAppDetail.id should echo the requested appBlockId').toBe(first.id);
// A human display name: manifest.name is the public allowlist field; fall back
// to blockId (the page does the same: name = manifest.name ?? blockId ?? id).
const detailName = detail!.manifest.name ?? detail!.blockId;
expect(
typeof detailName,
'detail should expose a display name (manifest.name or blockId)'
).toBe('string');
expect(detailName.length, 'the display name should be non-empty').toBeGreaterThan(0);
// The scopes + slots arrays are shape-correct (anon-display allowlist).
expect(Array.isArray(detail!.scopes), 'detail.scopes should be an array').toBe(true);
expect(
Array.isArray(detail!.manifest.targets ?? []),
'detail.manifest.targets should be an array (slot badges)'
).toBe(true);
// The DETAIL PAGE renders that block's name (host-rendered <Title> text). This
// proves the page's getAppDetail query + SSR appBlocks gate work end-to-end,
// not just the bare tRPC call. domcontentloaded ONLY.
const detailResp = await page.goto(`/apps/${encodeURIComponent(first.id)}`, {
waitUntil: 'domcontentloaded',
});
expect(detailResp?.status(), `GET /apps/${first.id} status`).toBeLessThan(400);
await expect(
page.getByRole('heading', { name: detailName }),
`the detail page should render the block name "${detailName}"`
).toBeVisible();
});
});