diff --git a/.claude/scheduled_tasks.lock b/.claude/scheduled_tasks.lock deleted file mode 100644 index 7122f92c0a..0000000000 --- a/.claude/scheduled_tasks.lock +++ /dev/null @@ -1 +0,0 @@ -{"sessionId":"d20ae04c-7892-42e4-ab3a-a2b4cde8bf3b","pid":107176,"procStart":"639141158551494140","acquiredAt":1778608378995} \ No newline at end of file diff --git a/.claude/skills/add-training-support/SKILL.md b/.claude/skills/add-training-support/SKILL.md new file mode 100644 index 0000000000..cb77e9eeca --- /dev/null +++ b/.claude/skills/add-training-support/SKILL.md @@ -0,0 +1,199 @@ +--- +name: add-training-support +description: Wire an existing ecosystem into the LoRA training system so it appears as a trainable base model in the training form. Adds the base-model entry, schema enums, orchestrator validation, feature flag, and per-ecosystem default params. Use when a model family (already in basemodel.constants.ts) needs training support — e.g. AI-Toolkit ecosystems like Anima, HiDream, Boogu. Always checks @civitai/client for the ecosystem's training input type first. +--- + +# Add Training Support + +Wires an existing ecosystem into the **training** form (the "Train a LoRA" flow), distinct from the generation form. After this, the ecosystem shows up as a selectable base model in Training Step 1 and submits a valid `training` step to the orchestrator. + +Training is a **separate subsystem from generation**. The generation handlers/graphs in `src/server/services/orchestrator/ecosystems/*` and `src/shared/data-graph/generation/*` are NOT part of this — do not touch them. The files below are the training path. + +## When to use + +- A new model family was just added via `add-ecosystem` and now needs to be trainable +- Re-enabling training for an ecosystem that was commented out +- The orchestrator/`@civitai/client` gained a new `*TrainingInput` type + +## Prerequisites + +The ecosystem, base model, family and license must already exist in +[basemodel.constants.ts](src/shared/constants/basemodel.constants.ts) (`ECO.`, +an `EcosystemRecord`, a `BaseModelRecord`). If any are missing, run **add-ecosystem** first. + +Almost every ecosystem added recently is **AI-Toolkit-based** (engine `'ai-toolkit'`, +mandatory). The worked reference is **`anima`** — it is the simplest complete example +(image, AI-Toolkit-only, no `modelVariant`). The single highest-leverage move when adding +a new ecosystem: **grep every file below for `anima` and add a parallel entry.** + +## Step 0 — Check `@civitai/client` for the training input type + +**Always** confirm the SDK ships the ecosystem's training type before wiring. It tells you +the exact field shape, whether `modelVariant` is required, and any fixed fields. + +```bash +grep -niE "AiToolkitTrainingInput|ecosystem: ''" \ + node_modules/@civitai/client/dist/generated/types.gen.d.ts +``` + +Note from the type: +- **`ecosystem` literal** — the exact string the orchestrator expects (e.g. `'boogu'`). +- **`modelVariant`** — present (e.g. flux1 `'dev'|'schnell'`, wan `'2.1'|'2.2'`) or absent. +- **Fixed/constrained fields** — e.g. Boogu declares `batchSize?: null | number` "Fixed at 1 + for this ecosystem". Honor these in the UI param bounds. +- **`readonly` outputs** (`defaultSteps`, `storageBuzzPerEpoch`, `maxBatchSize`, …) — server- + computed; never send them. + +If the type is missing in the installed version, check the latest (`npm view @civitai/client +versions --json | tail`) and bump with `pnpm add @civitai/client@`. If still absent, the +orchestrator dispatch (`training.orch.ts`) already casts `ecosystem as any`, so it works — but +prefer a typed SDK. + +## Step 1 — Gather the ecosystem's training defaults + +Get these from the user, the model card, or an orchestrator `whatif` sample request: + +- **Default steps** (the AI-Toolkit primary length knob — drives pricing) +- **lr** (unet LR), whether the **text encoder** is trained (usually disabled) +- **networkDim / networkAlpha**, **lrScheduler**, **optimizerType**, **noiseOffset** +- **batch size** bounds (many AI-Toolkit image ecosystems are fixed at 1) +- **resolution** (image ecosystems are typically 1024) +- The **base-model AIR**. If the model isn't on civitai yet, use the HF repository URN from the + sample as a placeholder and leave a comment to swap in + `urn:air::checkpoint:civitai:@` once uploaded. For AI-Toolkit-only + ecosystems this AIR is NOT sent to the orchestrator (it resolves the base model from the + ecosystem); it's used for UI display / `getTrainingFields.getModel`. + +Pick a stable **base-model key** (the `trainingModelInfo` key, e.g. `boogu`) and a +**baseType** (the `TrainingBaseModelType`, e.g. `'boogu'`). They can differ (SD has +`sd_1_5`/`anime`/… keys all mapping to baseType `sd15`), but for a single-checkpoint +ecosystem keep them the same. + +## Step 2 — Confirm the plan with the user + +``` +Adding training support for: (baseType: , key: ) +Engine: ai-toolkit (mandatory) Variant: +Defaults: steps , lr , dim/alpha /, scheduler , batch (max ), res +AIR: Feature flag: Training / -training (Flipt) +``` + +Wait for confirmation, then make all edits in one pass. + +## Step 3 — Edits + +### 3a. `src/utils/training.ts` (the core — ~7 spots) + +1. **`trainingBaseModelTypesImage`** (or `…Video` / `…Audio`) — add `''`. +2. **`aiToolkitStepDefault`** — add a branch returning the ecosystem's default steps if it + differs from 2000. +3. **`aiToolkitBatchMax`** — add a branch only if max batch > 1 (default returns 1). +4. **`trainingModelInfo`** — add the `: { label, pretty, type: '', description, + air, baseModel: '', isNew: true, aiToolkit: { ecosystem: '' + [, modelVariant] } }` entry. `baseModel` MUST match the `BaseModelRecord.name` / + ecosystem `key` in basemodel.constants.ts exactly (a mismatch produces a malformed AIR on + training completion and fails the post-train scan with 400 — see the inline comment on the + `hidream_o1` entry). +5. **`baseTypeToEcosystem`** — add `: ''`. +6. **`isAiToolkitSupported`** `supportedTypes` — add `''`. +7. **`isAiToolkitMandatory`** `mandatoryTypes` — add `''` (for AI-Toolkit-only + ecosystems). This auto-enables sample-prompt requirements and AI-Toolkit gating. +8. **`getDefaultEngine`** — add `if (baseType === '') return 'ai-toolkit';`. + +### 3b. `src/server/schema/model-version.schema.ts` + +- Add `export const trainingDetailsBaseModels = [''] as const;` next to the others. +- Spread `...trainingDetailsBaseModels` into the matching aggregate + (`trainingDetailsBaseModelsImage` / `…Video` / `…Audio`). +- The `trainingDetailsObj` zod enums (`baseModel`, `baseModelType`) derive from these + + `trainingBaseModelType`, so they update automatically. No `baseModelToTraningDetailsBaseModelMap` + entry unless mapping a `BaseModel` display name back to a key (Wan does this). + +### 3c. `src/server/schema/orchestrator/training.schema.ts` + +- Add a branch to the `aiToolkitTrainingParams` discriminated union: + ```ts + aiToolkitBaseParams.extend({ ecosystem: z.literal(''), modelVariant: z.undefined().optional() }), + ``` + (or `modelVariant: z.enum([...])` if the SDK type requires one). Without this, submission + validation rejects the new ecosystem. + +### 3d. `src/server/services/feature-flags.service.ts` + +- Add `Training: { availability: ['mod'], fliptKey: '-training' },`. Create the + Flipt flag too (use the `flipt` skill). Mod-only is the norm for a new/experimental base model. + +### 3e. `src/components/Training/Form/TrainingParams.tsx` + +- The `trainingSettings` array drives the UI and per-base defaults. Each setting's `overrides` + map is keyed by **base-model key** (``), not baseType. Grep the file for the reference + ecosystem's key (`anima:`) and add a parallel `: { all: { … } }` entry to **each** setting + where the new ecosystem should differ from the base default. For an AI-Toolkit image ecosystem + that's typically: `engine` (`'ai-toolkit'`), `maxTrainEpochs`, `trainBatchSize` (honor the + fixed/max from the SDK type), `targetSteps`, `saveEvery`, `resolution`, `shuffleCaption` + (disabled), `keepTokens` (disabled), `unetLR`, `textEncoderLR` (disabled), `lrScheduler`, + `minSnrGamma` (disabled), `networkDim`, `networkAlpha`, `noiseOffset`, `optimizerArgs`. + Skip a setting if the base default already matches (e.g. `optimizerType` AdamW8Bit, + `flipAugmentation` false for image). + +### 3f. `src/components/Training/Form/TrainingSubmitModelSelect.tsx` + +- Import `trainingDetailsBaseModels`. +- Add `const baseModel = !!formBaseModel && (trainingDetailsBaseModels as + ReadonlyArray).includes(formBaseModel) ? formBaseModel : null;`. +- Add a `{features.Training && (} + baseType="" makeDefaultParams={makeDefaultParams} isNew={…} />)}` block under the + right `mediaType` group (image/video/audio). Pick an unused `color`. +- Add `selectedRun.baseType === '' ||` to the "experimental build" alert condition + (for new/experimental base models). + +### 3g. `src/shared/constants/basemodel.constants.ts` + +- Add the training support entry to `ecosystemSupport`: + ```ts + { ecosystemId: ECO., supportType: 'training', modelTypes: loraOnly }, + ``` + (`loraOnly` is the standard for trained outputs — they're LoRAs.) Generation support is + independent; don't add it unless the ecosystem is also generatable. + +### Files that need NO change (generic / arbitrary-ecosystem aware) + +- `src/server/services/orchestrator/training/training.orch.ts` — `createTrainingStep_AiToolkit` + builds the input generically and casts `ecosystem as any`. Only add a branch if the ecosystem + needs special fields (sd1/sdxl send `model` + `minSnrGamma`; ACE-Step sends `samplesOverrides`). +- `src/store/training.store.ts` — `getDefaultTrainingParams` reads `trainingSettings` overrides + by key. Only touch if the new ecosystem should become a form default. +- `src/components/Training/Form/TrainingSubmit.tsx` / `TrainingSubmitAdvancedSettings.tsx` — + generic; resolve the ecosystem via `getAiToolkitEcosystem` once 3a is done. +- `src/server/common/enums.ts` `OrchEngineTypes` — `ai-toolkit` already present. + +## Step 4 — Typecheck + +```bash +pnpm run typecheck +``` + +Common failures: a `baseType`/key typo (the literal won't match the `TrainingBaseModelType` +union), a missing spread in the aggregate array, or a `baseModel` string that isn't a valid +`BaseModel`. Iterate until clean. + +## Step 5 — Verify (optional) + +With a dev server running (see `dev-server` skill) and the Flipt flag on for your user: +- Open the training form → Step 1 shows the new base model under its media type. +- Selecting it loads the expected default params in Step 3. +- The `whatif` submit returns a price without a validation error. + +## Notes + +- **Mandatory vs optional AI-Toolkit**: mandatory ecosystems (the common case) are gated solely + by their own `Training` flag. Optional ones (sd15/sdxl/flux/…) additionally check a + per-model `aiToolkit` flag via `aiToolkitFlagByBaseType` — only relevant if you're adding an + ecosystem that also supports Kohya. +- **Audio ecosystems** additionally gate on `audioTraining` and may send `samplesOverrides` + (see ACE-Step). **Video ecosystems** go in `trainingBaseModelTypesVideo` and usually disable + spatial params (resolution/clipSkip/noiseOffset). +- **Batch size**: if the SDK type fixes it (e.g. Boogu = 1), set the `trainBatchSize` override to + `{ all: { default: 1, min: 1, max: 1 } }` and leave `aiToolkitBatchMax` at its default. +- **Placeholder AIR**: fine to ship before the base model is on civitai — it's not sent for + AI-Toolkit-only ecosystems. Leave a comment so it gets swapped to the civitai URN later. diff --git a/.claude/skills/mod-actions/SKILL.md b/.claude/skills/mod-actions/SKILL.md index 63bc007244..f365a40599 100644 --- a/.claude/skills/mod-actions/SKILL.md +++ b/.claude/skills/mod-actions/SKILL.md @@ -39,6 +39,7 @@ cp .claude/skills/mod-actions/.env.example .claude/skills/mod-actions/.env | `reports.mjs` | Report handling | list, set-status, bulk-status, update, appeals, appeal-details, resolve-appeal | | `generation.mjs` | Generation moderation | flagged-consumers, flagged-reasons, consumer-strikes, review-strikes, user-generations, restrictions, resolve-restriction, allowlist-add, debug-audit, todays-counts, suspicious-matches | | `content.mjs` | Content & training | models, flagged-models, resolve-flagged, model-versions, rescan-model, restore-model, toggle-cannot-promote, toggle-cannot-publish, articles, training-models, approve-training, deny-training, mod-rule | +| `model3ds.mjs` | 3D Models moderation | list, get, files, unpublish, delete, restore, set-nsfw-level, toggle-tos, toggle-poi, toggle-minor, toggle-nsfw, toggle-unlisted | | `csam.mjs` | NCMEC/CSAM reporting | reports, stats, image-resources, create-report | --- @@ -233,6 +234,51 @@ node .claude/skills/mod-actions/content.mjs rescan-model 789 --- +## model3ds.mjs — 3D Models Moderation + +```bash +node .claude/skills/mod-actions/model3ds.mjs [options] +``` + +| Command | R/W | Description | +|---------|-----|-------------| +| `list` | R | List Model3Ds (`--status`, `--username`, `--limit`, `--cursor`) | +| `get ` | R | Get a Model3D by id | +| `files ` | R | List files attached to a Model3D | +| `unpublish ` | W | Set status -> Unpublished | +| `delete ` | W | Soft-delete (status -> Deleted, sets deletedAt/deletedBy) | +| `restore ` | W | Restore (Deleted -> Unpublished, Unpublished -> Published) | +| `set-nsfw-level ` | W | Override nsfwLevel (`--level [--lock]`) | +| `toggle-tos ` | W | Toggle tosViolation (locks the field) | +| `toggle-poi ` | W | Toggle poi (locks the field) | +| `toggle-minor ` | W | Toggle minor (locks the field) | +| `toggle-nsfw ` | W | Toggle nsfw (locks the field) | +| `toggle-unlisted ` | W | Toggle unlisted (locks the field) | + +Strikes against a Model3D use the existing `strikes.mjs` (the strike system is entity-type-agnostic): + +```bash +node .claude/skills/mod-actions/strikes.mjs create \ + --entity-type Model3D --entity-id --reason TOSViolation --points 1 +``` + +Common flags: +- `--json` — raw JSON output +- `--dry-run` — preview without making changes (write commands only) + +Examples: + +```bash +node .claude/skills/mod-actions/model3ds.mjs list --status Draft --limit 100 +node .claude/skills/mod-actions/model3ds.mjs get 42 +node .claude/skills/mod-actions/model3ds.mjs unpublish 42 +node .claude/skills/mod-actions/model3ds.mjs set-nsfw-level 42 --level 16 --lock +node .claude/skills/mod-actions/model3ds.mjs toggle-tos 42 --dry-run +node .claude/skills/mod-actions/model3ds.mjs restore 42 +``` + +--- + ## csam.mjs — NCMEC/CSAM Reporting ```bash diff --git a/.claude/skills/mod-actions/lib.mjs b/.claude/skills/mod-actions/lib.mjs index c84c9ab97f..e29fe65607 100644 --- a/.claude/skills/mod-actions/lib.mjs +++ b/.claude/skills/mod-actions/lib.mjs @@ -170,7 +170,8 @@ export function parseArgs(argv) { const key = arg.slice(2); // Boolean flags (no value) if (key === 'json' || key === 'dry-run' || key === 'confirm' || key === 'subtasks' || - key === 'flagged-for-review' || key === 'has-active-strikes' || key === 'force') { + key === 'flagged-for-review' || key === 'has-active-strikes' || key === 'force' || + key === 'lock') { flags[key] = true; } else { flags[key] = raw[++i]; diff --git a/.claude/skills/mod-actions/model3ds.mjs b/.claude/skills/mod-actions/model3ds.mjs new file mode 100644 index 0000000000..4ed256e225 --- /dev/null +++ b/.claude/skills/mod-actions/model3ds.mjs @@ -0,0 +1,226 @@ +#!/usr/bin/env node + +/** + * Model3D Moderation - CLI script for managing 3D models. + * + * Commands: + * list List Model3Ds (mod sees all statuses) + * get Get a Model3D by id + * files List files for a Model3D + * unpublish Unpublish (status -> Unpublished) + * delete Soft-delete (status -> Deleted, sets deletedAt/deletedBy) + * restore Restore: Deleted -> Unpublished, Unpublished -> Published + * set-nsfw-level Override nsfwLevel (use --level [--lock]) + * toggle-tos Toggle tosViolation (locks the field) + * toggle-poi Toggle poi (locks the field) + * toggle-minor Toggle minor (locks the field) + * toggle-nsfw Toggle nsfw (locks the field) + * toggle-unlisted Toggle unlisted (locks the field) + * + * Strikes + * Strikes work for EntityType.Model3D out of the box — file via the existing + * strikes.mjs: + * node strikes.mjs create --entity-type Model3D --entity-id ... + * + * Options: + * --json Output raw JSON + * --dry-run Preview without making changes (write commands) + * --status Filter `list` by status (Draft | Published | Unpublished | Deleted) + * --username Filter `list` by creator username + * --limit List limit (default: 50) + * --cursor List cursor for pagination (next cursor surfaces in JSON) + * --level NSFW level for set-nsfw-level (required) + * --lock Lock the field after set-nsfw-level (recommended on overrides) + */ + +import { + requireApiKey, trpcCall, parseArgs, output, intOrUndef, run, API_URL, +} from './lib.mjs'; + +requireApiKey(); + +const { command, target, flags } = parseArgs(process.argv); +const jsonMode = !!flags['json']; +const dryRun = !!flags['dry-run']; + +const VALID_STATUSES = ['Draft', 'Published', 'Unpublished', 'Deleted']; +const VALID_FLAG_FIELDS = ['tosViolation', 'poi', 'minor', 'nsfw', 'unlisted']; + +function showUsage() { + console.error(`Usage: node model3ds.mjs [options] + +Commands (READ): + list [--status ] [--username ] [--limit ] [--cursor ] + get + files + +Commands (WRITE): + unpublish + delete + restore + set-nsfw-level --level [--lock] + toggle-tos + toggle-poi + toggle-minor + toggle-nsfw + toggle-unlisted + +Options: + --json Output raw JSON + --dry-run Preview without making changes (write commands) + --status Draft | Published | Unpublished | Deleted + --username Filter list by creator username + --limit Default 50 + --cursor Cursor for pagination + --level NSFW level (required for set-nsfw-level) + --lock Lock the field after override + +Strikes against a Model3D are filed via strikes.mjs: + node strikes.mjs create --entity-type Model3D --entity-id ... + +Examples: + node model3ds.mjs list --status Draft --limit 100 + node model3ds.mjs get 42 + node model3ds.mjs unpublish 42 + node model3ds.mjs set-nsfw-level 42 --level 16 --lock + node model3ds.mjs toggle-tos 42 --dry-run +`); + process.exit(1); +} + +function requireId(label = 'ID') { + const id = parseInt(target); + if (!id || isNaN(id)) { + console.error(`Error: ${label} is required and must be a number`); + showUsage(); + } + return id; +} + +async function main() { + console.error(`Using API: ${API_URL}`); + + switch (command) { + // ── READ commands ────────────────────────────────────────── + + case 'list': { + const status = flags['status']; + if (status && !VALID_STATUSES.includes(status)) { + console.error(`Error: --status must be one of ${VALID_STATUSES.join(', ')}`); + showUsage(); + } + const input = { + limit: intOrUndef(flags['limit']) ?? 50, + cursor: intOrUndef(flags['cursor']), + statuses: status ? [status] : undefined, + username: flags['username'] || undefined, + }; + const result = await trpcCall('model3d.getInfinite', input, 'GET'); + output(result, jsonMode); + break; + } + + case 'get': { + const id = requireId('Model3D ID'); + const result = await trpcCall('model3d.getById', { id }, 'GET'); + output(result, jsonMode); + break; + } + + case 'files': { + const id = requireId('Model3D ID'); + const result = await trpcCall('model3d.getFiles', { id }, 'GET'); + output(result, jsonMode); + break; + } + + // ── WRITE commands ───────────────────────────────────────── + + case 'unpublish': { + const id = requireId('Model3D ID'); + if (dryRun) { + console.log(`[DRY RUN] Would unpublish Model3D ${id}`); + break; + } + const result = await trpcCall('model3d.unpublish', { id }, 'POST'); + output(result, jsonMode, () => `Unpublished Model3D ${id}`); + break; + } + + case 'delete': { + const id = requireId('Model3D ID'); + if (dryRun) { + console.log(`[DRY RUN] Would delete Model3D ${id}`); + break; + } + const result = await trpcCall('model3d.delete', { id }, 'POST'); + output(result, jsonMode, () => `Deleted Model3D ${id}`); + break; + } + + case 'restore': { + const id = requireId('Model3D ID'); + if (dryRun) { + console.log(`[DRY RUN] Would restore Model3D ${id}`); + break; + } + const result = await trpcCall('model3d.moderation.restore', { id }, 'POST'); + output(result, jsonMode, () => `Restored Model3D ${id}`); + break; + } + + case 'set-nsfw-level': { + const id = requireId('Model3D ID'); + const nsfwLevel = intOrUndef(flags['level']); + if (nsfwLevel === undefined) { + console.error('Error: --level is required'); + showUsage(); + } + const lock = !!flags['lock']; + const payload = { id, nsfwLevel, lock }; + if (dryRun) { + console.log(`[DRY RUN] Would set Model3D ${id} nsfwLevel=${nsfwLevel} lock=${lock}`); + break; + } + const result = await trpcCall('model3d.moderation.setNsfwLevel', payload, 'POST'); + output( + result, + jsonMode, + () => `Model3D ${id} nsfwLevel=${nsfwLevel}${lock ? ' (locked)' : ''}` + ); + break; + } + + case 'toggle-tos': + case 'toggle-poi': + case 'toggle-minor': + case 'toggle-nsfw': + case 'toggle-unlisted': { + const id = requireId('Model3D ID'); + const field = { + 'toggle-tos': 'tosViolation', + 'toggle-poi': 'poi', + 'toggle-minor': 'minor', + 'toggle-nsfw': 'nsfw', + 'toggle-unlisted': 'unlisted', + }[command]; + if (!VALID_FLAG_FIELDS.includes(field)) { + console.error(`Internal error: unmapped field for ${command}`); + process.exit(1); + } + if (dryRun) { + console.log(`[DRY RUN] Would toggle ${field} on Model3D ${id}`); + break; + } + const result = await trpcCall('model3d.moderation.toggleFlag', { id, field }, 'POST'); + output(result, jsonMode, () => `Toggled ${field} on Model3D ${id}`); + break; + } + + default: + console.error(`Unknown command: ${command}`); + showUsage(); + } +} + +run(main); diff --git a/.gitignore b/.gitignore index a4b8ba49dc..0b243dc211 100644 --- a/.gitignore +++ b/.gitignore @@ -132,6 +132,7 @@ scripts/ralph/progress.txt # Claude temp files tmpclaude-* /.claude/worktrees +.claude/scheduled_tasks.lock # Bundled SharedWorker scripts (generated by pnpm build:workers) /public/workers/ diff --git a/CLAUDE.md b/CLAUDE.md index 347f5cbb05..7fded7a066 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,6 +59,9 @@ pnpm test # Run Playwright tests pnpm run test:ui # Run tests with UI ``` +#### Never put unit tests under `src/pages` +Next.js 16 treats **every** `.ts`/`.tsx` file under `src/pages` (incl. nested `__tests__/`) as a route, and `next build` runs a route-type validator over it. A Vitest test file there fails the build with `Type '...test' does not satisfy the constraint 'ApiRouteConfig'. Property 'default' is missing` — and **only `next build` catches it**: `pnpm typecheck`, `pnpm test`/vitest, and the CI typecheck/unit/component tasks all pass, so it sneaks through to the preview `build-image` step. Keep handler tests in a `__tests__/` dir **outside** `src/pages` (e.g. `src/server/__tests__/`) and import the handler via the `~/pages/...` alias. (Bit us on PR #2653.) + ### Database ```bash pnpm run db:migrate:empty # Create an empty migration file diff --git a/docs/3d-models-diagram.md b/docs/3d-models-diagram.md new file mode 100644 index 0000000000..3921ece3f1 --- /dev/null +++ b/docs/3d-models-diagram.md @@ -0,0 +1,295 @@ +# 3D Models — Schema & Flow Diagrams + +Companion to `docs/3d-models-plan.md` (rev 9 — open questions resolved; ready to implement). Visual reference for the new entities, their relationships to existing Civitai tables, and the key user flows. + +Diagrams are Mermaid — render inline on GitHub, VS Code (with the **Markdown Preview Mermaid Support** extension), or any modern Markdown viewer. + +--- + +## 1. Entity Relationship Diagram + +**Legend**: blue = new tables, grey = existing tables we touch. Cardinality follows Mermaid's `||`, `|o`, `}o`, `}|` notation. + +```mermaid +erDiagram + Model3D ||--o{ Model3DFile : "has files (1 per format)" + Model3D ||--o{ TagsOnModel3D : "tagged with" + Model3D ||--o{ Model3DEngagement : "engaged by" + Model3D ||--o{ Model3DReport : "reports" + Model3D ||--o{ Model3DReview : "reviews" + Model3D ||--o| Model3DMetric : "metric" + Model3D ||--o| Thread : "1 discussion thread" + Model3D ||--o{ Post : "creator + community Posts" + Model3DReview ||--o| Post : "review image attachments" + Model3D ||--o{ CollectionItem : "in collections" + Model3D }o--|| Model3DLicense : "uses license" + Model3D }o--o| Image : "thumbnail (from generator)" + Model3D }o--o| Image : "source (image-to-3D)" + Model3D }o--|| User : "creator" + + Tag ||--o{ TagsOnModel3D : "tag of" + User ||--o{ Model3DEngagement : "engages" + User ||--o{ Model3DReview : "writes" + Report ||--o| Model3DReport : "discriminator" + Report ||--o| Model3DReviewReport: "discriminator" + Model3DReview ||--o| Thread : "review comments" + Thread ||--o{ CommentV2 : "comments" + + Model3D { + int id PK + citext name + text description + int userId FK + int thumbnailImageId FK "nullable @unique SetNull" + int licenseId FK + text licenseDetails "free-text when isCustom" + text workflowId UK "orchestrator; NULL = future upload" + int sourceImageId FK "image-to-3D source" + json generationParams "PolyGen input snapshot" + Model3DStatus status + bool nsfw + bool tosViolation + bool poi + bool minor + bool unlisted + text_array lockedProperties + Availability availability + int nsfwLevel + datetime publishedAt + } + + Model3DFile { + int id PK + int model3dId FK + text name + text url + float sizeKB "no cap in v1" + text format "lowercased glb fbx obj usdz stl" + bool isPrimary "at most one per Model3D" + json metadata + ScanResultCode virusScanResult + json rawScanResult + } + + Model3DLicense { + int id PK + text name UK + text description + bool allowCommercialUse + bool allowPrintFarm "CHECK requires commercial" + bool allowDerivatives + bool allowRedistribution + bool requireAttribution + bool isCustom + } + + Model3DReview { + int id PK + int model3dId FK + int userId FK + int rating "CHECK 1..5" + bool recommended + text details + bool nsfw + bool tosViolation + bool exclude + } + + Model3DReport { + int model3dId FK + int reportId FK + } + + Model3DReviewReport { + int model3dReviewId FK + int reportId FK + } + + Model3DEngagement { + int userId PK + int model3dId PK + Model3DEngagementType type "Favorite Hide Notify" + } + + Model3DMetric { + int model3dId PK + int downloadCount "sourced from ClickHouse" + int commentCount + int collectedCount + int tippedCount + int tippedAmountCount + int ratingCount + float ratingAvg + int recommendedCount + int reactionCount "denormalized from thumbnail ImageMetric" + int earnedAmount + int nsfwLevel "denormalized from Model3D" + int userId "denormalized" + Model3DStatus status "denormalized" + Availability availability "denormalized" + bool poi "denormalized" + bool minor "denormalized" + } + + TagsOnModel3D { + int model3dId PK + int tagId PK + datetime createdAt + } +``` + +**What's NOT here** (intentional omissions from rev 5): +- ~~`Model3DReaction`~~ — users react to the thumbnail `Image` (which is an existing `Image` row, reusing `ImageReaction`). +- ~~`Model3DDownloadHistory`~~ — download events go to ClickHouse; the rollup denormalizes into `Model3DMetric.downloadCount`. +- ~~`Model3DVersion`~~ — no versioning in v1. + +--- + +## 2. Existing-table touch points + +```mermaid +flowchart LR + Model3D[(Model3D)]:::new + Model3DFile[(Model3DFile)]:::new + Model3DLicense[(Model3DLicense)]:::new + Model3DReview[(Model3DReview)]:::new + + User[User]:::existing + Image[Image]:::existing + Tag[Tag]:::existing + Post[Post]:::existing + Thread[Thread]:::existing + CommentV2[CommentV2]:::existing + Report[Report]:::existing + CollectionItem[CollectionItem]:::existing + BuzzTip[BuzzTip]:::existing + ImageReaction[ImageReaction]:::existing + Meilisearch[(Meilisearch model3d index)]:::external + ClickHouse[(ClickHouse events)]:::external + S3[(S3 3d/ prefix)]:::external + Orchestrator[(Orchestrator PolyGen)]:::external + + Orchestrator -- "submitWorkflow PolyGenStep async" --> Model3D + Orchestrator -- "model.url, fbxModel.url" --> S3 + S3 -- "url" --> Model3DFile + + User -- "creator FK" --> Model3D + Image -- "thumbnailImageId FK" --> Model3D + Image -- "sourceImageId FK (image-to-3D)" --> Model3D + ImageReaction -. "reactions ride on thumbnail Image" .-> Model3D + Model3DLicense -- "licenseId FK" --> Model3D + Model3D -- "files" --> Model3DFile + + Tag -- "TagsOnModel3D join" --> Model3D + Model3D -- "model3dId column" --> Thread + Thread -- "comments" --> CommentV2 + Model3D -- "model3dId column" --> Post + Model3D -- "Model3DReport discriminator" --> Report + Model3DReview -- "Model3DReviewReport discriminator" --> Report + Model3D -- "model3dId column + unique extended" --> CollectionItem + BuzzTip -. "entityType='Model3D' in schema enum" .-> Model3D + Model3D -. "indexed" .-> Meilisearch + ClickHouse -. "download events aggregated into Model3DMetric.downloadCount" .-> Model3D + + classDef new fill:#1d4ed8,stroke:#1e3a8a,color:#fff; + classDef existing fill:#374151,stroke:#1f2937,color:#fff; + classDef external fill:#7c2d12,stroke:#431407,color:#fff,stroke-dasharray:3 3; +``` + +Key points: + +- **Reactions reuse `ImageReaction`** on the thumbnail Image — no Model3D-specific reaction table. +- **Downloads go to ClickHouse** as events, not a Postgres table. `Model3DMetric.downloadCount` is a denormalized aggregate. +- **Source of v1 content** is the orchestrator's PolyGen recipe — no upload path in v1. +- **`Image` has two FKs into `Model3D`**: thumbnail (from PolyGen output) and source (for image-to-3D inputs). + +--- + +## 3. Generate + publish lifecycle + +```mermaid +flowchart TD + Start([User opens Generate panel, picks 3D Model]) --> A1[Form: textTo3D or imageTo3D
+ Meshy params: prompt, topology,
polycount, symmetry, PBR, seed] + A1 --> A1a{Image-to-3D?} + A1a -- yes --> A1b[Ingest source image as Image row first
mirroring Sora sourceImageSchema] + A1a -- no --> A2 + A1b --> A2[Submit via submitWorkflow + PolyGenStep
type='polyGen' async] + A2 --> A3[Orchestrator runs Meshy via Fal
user sees queue card with status] + A3 --> A4[Workflow result handler fires server-side:
PolyGenOutput model GLB + optional FBX + thumbnail] + + A4 --> B1[Copy blobs to S3 3d/ prefix
handle nullable url + expiry] + B1 --> B2[Ingest thumbnail as Image row
NSFW + CSAM scan via standard pipeline] + B2 --> B3[Create Model3D Draft
workflowId set UNIQUE
generationParams snapshot] + B3 --> B4[Create Model3DFile rows
one per normalized format
GLB isPrimary] + B4 --> B5[Queue card now shows thumbnail
not inline WebGL] + + B5 --> C1{User clicks 'Post from Generation'?} + C1 -- no --> Stay([Model3D stays Draft
no public surface]) + C1 -- yes --> D1[Mutation creates empty Post
redirect to /posts/:id/edit] + D1 --> D2[Edit page hosts Post + Model3D fields:
name description tags license NSFW] + D2 --> D3[Publish: Model3D.status = Published
Post.model3dId set + flipped to public] + D3 --> D4[Meilisearch indexed
Model3DMetric initialized] + D4 --> End([/3d-models/:id live]) + + style A2 fill:#1d4ed8,color:#fff + style B3 fill:#1d4ed8,color:#fff + style D1 fill:#1d4ed8,color:#fff + style End fill:#15803d,color:#fff +``` + +--- + +## 4. Detail page read path + +```mermaid +flowchart LR + Req([GET /3d-models/:id]) --> SVC[model3d.service.getById] + + SVC --> Q1[Model3D + license + thumbnailImage + sourceImage] + SVC --> Q2[Model3DFile rows ordered isPrimary first] + SVC --> Q3[Model3DMetric] + SVC --> Q4[TagsOnModel3D + Tag] + SVC --> Q5[Thread + paginated CommentV2] + SVC --> Q6[Post where model3dId = :id
creator + community 'Makes/Uses'] + SVC --> Q7[Model3DReview count + avg rating] + + Q1 --> Render[Detail page render] + Q2 --> Render + Q3 --> Render + Q4 --> Render + Q5 --> Render + Q6 --> Render + Q7 --> Render + + Render --> V1[3D Viewer
three.js + GLTFLoader
dynamic-imported, GLB primary] + Render --> V2[Files dropdown
signed URLs
download events to ClickHouse] + Render --> V3[Generation Details panel
prompt, topology, polycount, seed] + Render --> V4[Comments + reactions on thumbnail] + Render --> V5[Makes & Uses rail] + Render --> V6[Reviews summary
link to /3d-models/:id/reviews] +``` + +--- + +## 5. Cross-reference + +| Entity | Existing analog | Difference | +| ------------------------- | ---------------------- | --------------------------------------------------------------------- | +| `Model3D` | `Model` | No `baseModel`/`ecosystem`/`ModelType`/versioning. Has `workflowId`. | +| `Model3DFile` | `ModelFile` | `format String` (not enum), `isPrimary`, `(model3dId, format)` unique | +| `Model3DLicense` | `License` | Adds `allowPrintFarm`, `allowRedistribution`, `isCustom` | +| `Model3DReview` | `ResourceReview` | Scoped to `model3dId` (no `modelVersionId`) | +| `Model3DReport` | `ModelReport` | Identical shape | +| `Model3DEngagement` | `ModelEngagement` | Drops `Mute` from the enum | +| `Model3DMetric` | `ModelMetric` | `downloadCount` sourced from ClickHouse; adds rating fields | +| `TagsOnModel3D` | `TagsOnModels` | Identical shape | +| `Model3D` ↔ `Thread` | `Model` ↔ `Thread` | Wide-FK columns added (`model3dId` + `model3dReviewId`) | +| `Model3D` ↔ `Post` | `ModelVersion` ↔ `Post`| Wide-FK column added (`Post.model3dId`) | +| `Model3D` ↔ `CollectionItem` | `Model` ↔ `CollectionItem` | New FK column + extended unique constraint | +| ~~`Model3DReaction`~~ | ~~`ImageReaction`~~ | **Removed** — react on the thumbnail Image instead | +| ~~`Model3DDownloadHistory`~~ | ~~`DownloadHistory`~~ | **Removed** — ClickHouse events; rollup in `Model3DMetric` | +| ~~`Model3DFileType` enum~~| ~~`ModelFile.type`~~ | **Removed** — `format String` for flexibility | + +--- + +**Source of truth**: `prisma/schema.full.prisma` (the actual editable Prisma schema; `prisma/schema.prisma` is auto-generated from it via `scripts/generate-slim-schema.js`). Migration SQL is hand-written in `prisma/migrations/20260526120000_add_3d_models/migration.sql` to mirror the schema changes; per CLAUDE.md, it's applied manually rather than via `prisma migrate deploy`. If diagrams drift from the schema, the schema wins. diff --git a/docs/3d-models-followups.md b/docs/3d-models-followups.md new file mode 100644 index 0000000000..9e810a59d6 --- /dev/null +++ b/docs/3d-models-followups.md @@ -0,0 +1,271 @@ +# 3D Models — Phase 2 Follow-ups (profile feed + moderation) + +## M1 — Profile-page 3D Models tab + +### Current state (research findings) + +- The profile shell is `src/components/Profile/ProfileLayout2.tsx`. It wraps each `/user/[username]/*` page and renders `` as the subNav. +- `ProfileNavigation.tsx:38-86` is a single static map of `opts` keyed by `'[username]'`, `models`, `posts`, `images`, `videos`, `articles`, `comics`, `collections`. There is **no registry** — adding a tab means adding a key to that object. +- Each tab page in `src/pages/user/[username]/*.tsx` (e.g. `models.tsx`, `posts.tsx`, `articles.tsx`) follows the same pattern: + - `getServerSideProps` via `createServerSideProps({ useSSG: true, ... })`, with optional feature-flag redirect (`articles.tsx:28-34` redirects to `/user/[username]` when `!features.articles`). + - SSG prefetches `userProfile.get` + `userProfile.overview`. + - Page reads `currentUser` + `query.username`, derives `selfView` via `postgresSlugify(currentUser.username) === postgresSlugify(username)`. + - Renders an optional `FeedContentToggle` (published/draft) **only for self-view**, then a `MasonryContainer` with the relevant `*Infinite` component. + - Default export wraps `Page(..., { getLayout: UserProfileLayout })` so the `ProfileLayout2` shell + nav are inherited. +- Counts are surfaced by `trpc.userProfile.overview` → `getUserContentOverviewHandler` → `getUserContentOverview` in `src/server/services/user-profile.service.ts:28-62` → Redis composite cache `userContentOverviewCache` (`src/server/redis/caches.ts:896-1066`). + - `UserContentOverview` type (`caches.ts:896-908`) hard-codes the keys: `modelCount`, `imageCount`, `videoCount`, `postCount`, `articleCount`, `bountyCount`, `bountyEntryCount`, `collectionCount`, `comicCount`, `hasReceivedReviews`. **No `model3dCount` field.** + - There are three variants (all / sfw / public) each materialized via per-table cache factories at `caches.ts:503-520`, each backed by a single counting query. +- `trpc.model3d.getInfinite` already accepts `username`, `userId`, and `includeDrafts` (`src/server/schema/model3d.schema.ts:54-64`). The service (`model3d.service.ts:192-259`) gates non-mod / non-owner reads correctly — `allowDrafts` is keyed on `userId === user.id`, so the profile page should pass `userId: profileUser.id` (or `username`) **and** `includeDrafts` for self-view. + - Caveat: `model3d.service.ts:210-219` returns an empty list for non-mods unless `user` is signed in **and** (no `userId` filter, OR `userId === user.id`, OR `username` is set). Passing `username` is the safe path — that branch is allowed for any signed-in user. Without a session, anonymous viewers also get an empty list. **This means the new tab will be empty for logged-out visitors until the `model3dFeed` flag opens past mod-only,** which matches the launch plan. +- `Model3DCard.tsx` (`/home/luis_rojas/Work/civitai/src/components/Cards/Model3DCard.tsx`) is the existing card; consumes `inferRouterOutputs['model3d']['getInfinite']['items'][number]`. Already in use by `/3d-models/index.tsx`. +- `ProfileLayout2.tsx:48-79` builds a per-subpage `deIndex` map (`subpageCounts`) used for SEO no-index decisions. It will need a `'3d-models'` entry once we add the route. +- `model3dFeed` flag is mod-only at launch (`feature-flags.service.ts:203`). Profile tab gating mirrors `articles.tsx:28-34` — redirect to `/user/[username]` when off. + +### File touch list + +- `src/components/Profile/ProfileNavigation.tsx` — add `'3d-models'` entry to `opts`, gate `disabled` on `features.model3dFeed`. +- `src/pages/user/[username]/3d-models.tsx` — **new file**, mirrors `articles.tsx` structure. +- `src/components/Profile/ProfileLayout2.tsx` — extend `subpageCounts` map at `:66-75` so the SEO no-index logic understands `'3d-models'`. +- `src/server/redis/caches.ts` — add `userModel3DCount{,Sfw,Public}Cache` factories (mirror of `userPostCountCache` at `:570-613`), add `model3dCount` to `UserContentOverview` type at `:896-908`, plumb through `mergeOverviewResults` + all three `getUserContentOverview*` functions + `userContentOverviewCache.refresh` (`:1068-1100ish`). +- `src/server/services/user-profile.service.ts` — no change (it just forwards the cache result). +- `src/components/Profile/ProfileSidebar.tsx` / `ProfileSectionsSettingsInput.tsx` — **optional**: check whether they enumerate content types for the "configure sections" UI; only touch if user-visible. +- `src/server/redis/cache-invalidation.ts` or equivalent — wire `userContentOverviewCache.refresh(userId)` on Model3D publish/unpublish/delete (search for where `Model.publish` triggers `userModelCountCache.refresh`). + +### Phased steps + +**Phase 1 — Backend count plumbing (~2-3 h)** + +1. In `src/server/redis/caches.ts`, add three new `createUserContentCountCache` instances for `model3dCount`, `model3dCount:sfw`, `model3dCount:public` keyed off `"Model3D"`. Query template: + ```sql + SELECT "userId" as id, COUNT(*)::INT as "model3dCount" + FROM "Model3D" + WHERE "userId" IN (...) + AND "status" = 'Published' + AND "deletedAt" IS NULL + AND availability != 'Private' + -- sfw/public variants: AND ("nsfwLevel" & ${flag}) != 0 + GROUP BY "userId" + ``` +2. Extend `UserContentOverview` type with `model3dCount: number`. +3. Update `mergeOverviewResults`, all three `getUserContentOverview*` aggregator functions, and the `userContentOverviewCache.refresh` block to include the new caches. +4. Wire `userContentOverviewCache.refresh(userId)` into `publishModel3D` / `unpublishModel3D` / `deleteModel3D` in `src/server/services/model3d.service.ts` (look at how `model.service.ts` does this for `Model.publish`). +5. **Verify** SSR prefetch (`ssg?.userProfile.overview.prefetch`) returns the new field without changes — it should, since the procedure returns the cache shape as-is. + +**Phase 2 — Profile tab page (~2 h)** + +6. Create `src/pages/user/[username]/3d-models.tsx`. Use `articles.tsx` as the template: + - `getServerSideProps`: `if (!features?.model3dFeed) return { redirect: { destination: '/user/${username}', permanent: false } };` + banned-user redirect + SSG prefetch `userProfile.get` and `userProfile.overview`. + - Use `useCurrentUser()` + `postgresSlugify` to derive `selfView`. Compute `isMod = currentUser?.isModerator`. + - State: `[section, setSection] = useState<'published' | 'draft'>(selfView ? ... : 'published')`. Show `FeedContentToggle` only when `selfView || isMod`. + - Render `trpc.model3d.getInfinite.useInfiniteQuery({ limit: 50, username, includeDrafts: section === 'draft' && (selfView || isMod) })`. + - Use the same Masonry + `Model3DCard` + `InViewLoader` pattern from `/pages/3d-models/index.tsx`. (Consider extracting that block into `src/components/Model3D/Infinite/Model3DsInfinite.tsx` for reuse between the global feed and the profile tab — see Risks.) + - Empty state: reuse ``; if `selfView`, add a CTA linking to `/3d-models/create` (or the wizard route — confirm with `Model3DGenerationForm.tsx` location). + - Default export `Page(UserModel3DsPage, { getLayout: UserProfileLayout })`. + +**Phase 3 — Navigation entry (~30 min)** + +7. In `ProfileNavigation.tsx`, add to `opts`: + ```ts + '3d-models': { + url: `${baseUrl}/3d-models`, + icon: (props) => , + count: userOverview?.model3dCount ?? 0, + disabled: !features.model3dFeed || !!user?.bannedAt, + }, + ``` + `IconCube` is already used (e.g. `Model3DCard.tsx:2`). Confirm import. +8. In `ProfileLayout2.tsx:66-74`, add `'3d-models': overview?.model3dCount` to `subpageCounts` so SEO de-index keeps working for empty 3D tabs. +9. Confirm `activePath = router.pathname.split('/').pop()` (`ProfileNavigation.tsx:35`) resolves to `'3d-models'` for `/user/[username]/3d-models`. Yes — `split('/').pop()` on that route returns the literal `'3d-models'`. The `opts` key must therefore be the literal string `'3d-models'` (hyphenated, not camelCase). + +**Phase 4 — Polish (~1 h)** + +10. If the moderator wants to see a creator's drafts even when they're not the owner, broaden the gate: in `UserModel3DsPage`, allow `selfView || isMod` for the draft toggle, **but** the service's `allowDrafts` check (`model3d.service.ts:230`) only honors `includeDrafts` when `userId === user.id`. Either (a) extend the service to also honor `includeDrafts` when `user.isModerator`, or (b) pass `statuses: [Draft, Published, Unpublished]` for mods (the service allows any `statuses` for `isModerator`). Option (b) is simpler. +11. Add `/user/[username]/3d-models` to any sitemap or canonical-URL helpers (grep for the existing `/user/[username]/models` reference in `src/pages/sitemap*` if it exists). + +### Risks / open questions + +- **R1**: `Model3DsInfinite` duplication. The masonry block in `pages/3d-models/index.tsx:42-94` would be re-implemented in the profile page. Cleaner: extract a `Model3DsInfinite` component (mirror of `ModelsInfinite`) that takes filter props. Adds ~30 min of refactor but avoids drift. Recommend doing this in Phase 2. +- **R2**: Anonymous viewers will see zero 3D models because the service short-circuits non-signed-in non-mod requests (`model3d.service.ts:210-218`). This is consistent with launch gating but means the profile tab badge count may show `model3dCount > 0` while the tab itself is empty for anon viewers. Decide whether to: + - hide the tab entirely for anon viewers (`disabled: !features.model3dFeed || !currentUser`), or + - relax the service gate now that the public feed is on its own flag-gated route. **Recommend**: only hide the count badge when anon, but keep the tab clickable so the empty state explains the gate. +- **R3**: Banned-user gate in service is `userId && userId !== user.id && !username` — passing `username` bypasses this for any signed-in user. If profile owner is banned but `username` is passed, the service will still return their content. The page-level redirect at `/user/[username]/3d-models` should mirror `models.tsx:36-39` (redirect when `user?.bannedAt`). +- **R4**: `model3dCount` cache invalidation must run on publish/unpublish/delete/upsert (when status changes). Grep `userModelCountCache.refresh` to find the existing pattern in `model.service.ts` and mirror it. +- **R5**: `availability` column exists on Model3D (`schema.full.prisma:5810`) defaulting to `Public`. Private Model3Ds (if/when that's enabled) should be excluded from the count — query mirrors the Model pattern. +- **Open Q**: should there be a "Private" segment for Model3D drafts (per `models.tsx:67-69, 192`)? Defer to Phase 3; the entity has `availability` but no current product flow sets it to `Private`. + +--- + +## M2 — Moderation integration + +### Current state (research findings) + +- **Report enum + schema is wired**: `ReportEntity.Model3D` + `Model3DReview` exist (`src/shared/utils/report-helpers.ts:15-16`); `reportTypeNameMap` and `reportTypeConnectionMap` cover both (`src/server/services/report.service.ts:123-124, 141-142`); the `Report` table has `model3d` / `model3dReview` relations (`prisma/schema.full.prisma:1334-1335, 5884-5926`). +- **But `report.create` will fail for Model3D** until: (a) `report.controller.ts` `getReports` select gets a `model3d` + `model3dReview` block, and (b) the `ReportModal` adds `ReportEntity.Model3D` to each form's `availableFor`. The detail page already has a TODO at `pages/3d-models/[id]/[[...slug]].tsx:255-269` blocking the report button on this. +- **Two parallel report paths exist**: + - Centralized: `trpc.report.create` → `createReportHandler` → `createReport` in `report.service.ts:150-303`. This goes through the moderator queue (`trpc.report.getAll` → `getReportsHandler` in `report.controller.ts:100-271`). + - Direct: `trpc.model3d.reports.createForModel` → `createModel3DReport` in `model3d-report.service.ts:42-77`. Writes to the same `Report` table but bypasses `report.service.createReport`'s tag-vote / NSFW-action side effects. + - **Recommendation**: route all UI through `trpc.report.create` so NSFW / TOS / CSAM side-effects fire correctly. Keep the direct procedures for programmatic / SDK callers (or deprecate them). +- **`getReportsHandler` does not select model3d join data** (see `report.controller.ts:113-245`). The moderator queue page (`src/pages/moderator/reports.tsx`) iterates `Object.values(ReportEntity)` for the segmented control (`reports.tsx:226`), so the `Model3D` / `Model3DReview` tabs **already render** but produce empty results — the `where` filter `{ type: { isNot: null } }` at `report.service.ts:319-321` works (because the relation exists), but the row data needed by `getReportLink` (`reports.tsx:442-464`) and `ReportDrawer` is missing. +- **`getReportLink`** at `reports.tsx:442-464` has no case for `report.model3d` / `report.model3dReview`. Without it, the "Open reported item" button at `reports.tsx:131-141` will have an undefined href. +- **Model3D content actioning**: + - `trpc.model3d.unpublish` exists (`model3d.router.ts:111`) — non-destructive, status → Unpublished. + - `trpc.model3d.delete` exists (`model3d.router.ts:115`) — soft delete (status → Deleted, sets `deletedAt`, `deletedBy`). + - **No** `setNsfwLevel`, `setTosViolation`, `setPoi`, `setMinor`, `lockedProperties` mutation endpoint. `Model3D.nsfwLevel` is derived from the thumbnail Image's `nsfwLevel` via a batch job (per the plan; see `model3d.service.ts:82-106` selects but no setter). For Model — for comparison — moderators flip these via `model.toggleCannotPublish` / `model.toggleCannotPromote` / report-action side effects. Model3D has no equivalents. + - `lockedProperties: string[]` exists on Model3D (`schema.full.prisma:5809`) and the upsert service strips locked props for non-mods (`model3d.service.ts:286-291`). Mods can therefore set `tosViolation` / `nsfw` / `nsfwLevel` via `trpc.model3d.upsert` already — but there's no convenient UI surface and it's not idempotent against `lockedProperties`. +- **Strike system**: `EntityType.Model3D` exists in the enum (`schema.full.prisma:3639`), and `UserStrike.entityType` is `EntityType?` (`schema.full.prisma:5544`) — so strikes can already reference Model3D with no schema change. `createStrike` in `strike.service.ts:470-525` just passes `entityType` through; no per-type switch. **Strikes work out of the box once a UI calls `trpc.strike.create` with `entityType: 'Model3D'` and an `entityId`.** +- **Appeals**: `Appeal.entityType` is also the same `EntityType` enum. **However**, `createEntityAppealHandler` in `src/server/controllers/report.controller.ts:286-318` has a hard-coded switch: + ```ts + case EntityType.Image: + ... + default: + throw throwDbCustomError('Entity type not supported for appeals'); + ``` + So **appeals only work for Image today** despite the schema supporting more. Adding Model3D to that switch is trivial (lookup ownership via `model3d.findUnique`). +- **`mod-actions` skill** (`.claude/skills/mod-actions/`): no references to `model3d` / `Model3D` anywhere in the skill scripts. The closest analog is `content.mjs` which exposes `models query / flagged-models / restore / toggleCannotPublish`. There is no Model3D module yet. +- **No `/moderator/model3ds` page exists**. The existing flow assumes the report queue + the thumbnail-image affordance (`Model3DModAction.tsx`) are sufficient. +- **`Model3DModAction.tsx`** only surfaces an "unpublish" action. No "delete", "set NSFW", "set TOS violation", or "issue strike" affordance. + +### File touch list + +- `src/server/controllers/report.controller.ts` — extend the `select` block in `getReportsHandler` (`:104-246`) with `model3d` + `model3dReview` joins; extend the `items.map` projection (`:249-265`) to surface them. +- `src/pages/moderator/reports.tsx` — extend `getReportLink` (`:442-464`) with `model3d` (returns `/3d-models/${report.model3d.id}`) and `model3dReview` (returns `/3d-models/${report.model3dReview.model3dId}/reviews`). +- `src/components/Modals/ReportModal.tsx` — add `ReportEntity.Model3D` to `availableFor` arrays for `TOSViolation`, `AdminAttention`, `Spam` (mirror the Article / Post entries at `:53-60, 67-79, 86-99, 117-131`). For NSFW reports, add `ReportEntity.Model3D` to the `ArticleNsfwForm` group (`:53-60`) since Model3D NSFW is content-level, not tag-level like Image. +- `src/components/Modals/ReportModal.tsx` — extend `onSuccess` switch (`:196-234`) with a `ReportEntity.Model3D` branch that invalidates `trpc.model3d.getById` + `trpc.model3d.getInfinite`. +- `src/pages/3d-models/[id]/[[...slug]].tsx` — replace the TODO block at `:255-269` with `openReportModal({ entityType: ReportEntity.Model3D, entityId: model3d.id })`. +- `src/components/Modals/ReportModal.tsx` — handle `entityType === ReportEntity.Model3D` for the `useVoteForTags` call site (`:172`) — Model3D has tags via `TagsOnModel3D`, but there's no current vote pipeline. Pass `undefined` or guard the call. +- `src/server/services/report.service.ts` — extend the `createReport` switch (`:194-269`) with Model3D side effects: NSFW report → set `nsfw: true`; TOS report → no engagement table (skip); CSAM report → log only (3D doesn't have an `ingestion` column). +- `src/server/services/model3d.service.ts` — **new** moderator setters: `setModel3DNsfwLevel({ id, nsfwLevel })`, `toggleModel3DTosViolation({ id })`, `toggleModel3DPoi({ id })`, `toggleModel3DMinor({ id })`, plus invalidation of `userContentOverviewCache.refresh(model3d.userId)` on status changes. +- `src/server/routers/model3d.router.ts` — wire the new mod-only mutations under a `moderation` sub-router (mirror of `reviews`/`reports` style) using `moderatorProcedure`. +- `src/components/Model3D/Moderation/Model3DModAction.tsx` — extend to surface "unpublish", "delete", "toggle TOS", "set NSFW level" — or build a sibling `Model3DModMenu` for the detail page. +- `src/server/controllers/report.controller.ts` — extend `createEntityAppealHandler` switch (`:296-305`) with a `case EntityType.Model3D` branch (ownership lookup via `dbRead.model3D.findUnique`). +- `.claude/skills/mod-actions/content.mjs` — add `model3ds`, `model3d-restore`, `model3d-unpublish`, `model3d-toggle-tos` commands; OR add a new `.claude/skills/mod-actions/model3ds.mjs` script. Update `SKILL.md`'s file table at `:32-43`. + +### Phased steps + +**Phase 1 — Report queue surface (~2 h)** + +1. In `report.controller.ts:113-246`, add `model3d` and `model3dReview` to the `select` block: + ```ts + model3d: { + select: { + model3d: { select: { + id: true, name: true, nsfw: true, tosViolation: true, + thumbnailImage: { select: { id: true, url: true, name: true } }, + user: { select: simpleUserSelect }, + }}, + }, + }, + model3dReview: { + select: { + model3dReview: { select: { + id: true, model3dId: true, rating: true, nsfw: true, tosViolation: true, + user: { select: simpleUserSelect }, + }}, + }, + }, + ``` +2. Extend the `items.map` projection at `:249-265` with `model3d: item.model3d?.model3d, model3dReview: item.model3dReview?.model3dReview`. +3. In `pages/moderator/reports.tsx`, extend `getReportLink` (`:442-464`) and confirm the `Model3D` / `Model3DReview` segmented-control tabs render correctly. +4. Smoke-test: file a report against an existing Model3D via `trpc.model3d.reports.createForModel`, then load `/moderator/reports` and switch the segmented control to `Model3d` (case-sensitivity check — `upperFirst('model3d')` is `'Model3d'`, fine). + +**Phase 2 — Centralized report flow (~1.5 h)** + +5. In `ReportModal.tsx`, add `ReportEntity.Model3D` to: + - NSFW form: probably reuse `ArticleNsfwForm` (`:49-61`) since Model3D doesn't have per-tag NSFW votes. + - TOSViolation, AdminAttention, Spam: add to existing arrays (`:67-79, 86-99, 117-131`). +6. Guard the `useVoteForTags` call (`:172`) so it's a no-op for `ReportEntity.Model3D` (Model3D has tags but no rating-request pipeline yet). +7. Extend the `onSuccess` switch (`:196-234`) with a Model3D case that invalidates `model3d.getById` and `model3d.getInfinite`. +8. Replace the TODO block in `pages/3d-models/[id]/[[...slug]].tsx:255-269` with `openReportModal({ entityType: ReportEntity.Model3D, entityId: model3d.id })`. +9. In `report.service.ts:194-269`, add Model3D NSFW / TOS side effects (mirror `case ReportEntity.Post: tx.post.update({ where: { id }, data: { nsfw: true }})` at `:251-253`). +10. **Deprecate or document**: `trpc.model3d.reports.createForModel` is now redundant with `trpc.report.create`. Either remove it (after auditing callers) or document it as the "SDK-only" path. Recommend keeping for now, but updating UI callers to use `trpc.report.create`. + +**Phase 3 — Content actioning endpoints (~2 h)** + +11. In `src/server/schema/model3d.schema.ts`, add: `setModel3DNsfwLevelSchema`, `toggleModel3DFlagSchema` (`{ id, field: 'tosViolation' | 'poi' | 'minor' | 'nsfw' }`). +12. In `src/server/services/model3d.service.ts`, implement: + ```ts + setModel3DNsfwLevel({ id, nsfwLevel, user }) // mod-only; also locks `nsfwLevel` in lockedProperties + toggleModel3DFlag({ id, field, user }) // mod-only; flips the boolean, locks the field + ``` + Both refresh the user content overview cache and queue search-index updates if/when 3D models hit Meilisearch. +13. In `src/server/routers/model3d.router.ts`, add a `moderation` sub-router: + ```ts + moderation: router({ + setNsfwLevel: moderatorProcedure.input(setModel3DNsfwLevelSchema).mutation(...), + toggleFlag: moderatorProcedure.input(toggleModel3DFlagSchema).mutation(...), + }) + ``` + (Not flag-gated — mods always have access.) +14. Extend `Model3DModAction.tsx` with a dropdown menu offering: Unpublish, Delete, Toggle NSFW, Toggle TOSViolation, Set NSFW Level, Open in moderator view. OR build a dedicated `` for the detail page that surfaces these only for `currentUser?.isModerator`. + +**Phase 4 — Strikes + appeals (~1 h)** + +15. Strikes already work for `EntityType.Model3D` since the schema and service are entity-type-agnostic. Verify by calling `trpc.strike.create` with `entityType: 'Model3D', entityId: ` via the `mod-actions` skill once Phase 3 lands. No code change. +16. **Appeals**: in `src/server/controllers/report.controller.ts:296-305`, add: + ```ts + case EntityType.Model3D: + const m3d = await dbRead.model3D.findUnique({ + where: { id: input.entityId }, + select: { userId: true }, + }); + if (!m3d) throw throwNotFoundError('3D model not found'); + if (m3d.userId !== userId) throw throwAuthorizationError(); + break; + ``` +17. Surface "Appeal" CTA on the Model3D detail page for unpublished/deleted-by-mod states (mirror the Image appeal flow — find via `grep -rn 'createEntityAppeal\|appealable' src/components/Image/`). + +**Phase 5 — Skill integration (~1.5 h)** + +18. Add a new file `.claude/skills/mod-actions/model3ds.mjs` (mirror `content.mjs` layout) with commands: + - `list` → `trpc.model3d.getInfinite` (mod sees all statuses) + - `get ` → `trpc.model3d.getById` + - `unpublish ` / `delete ` / `restore ` (need to add a `restoreModel3D` endpoint too — flip status from Unpublished back to Published; trivial parallel to unpublish) + - `set-nsfw-level --level ` / `toggle-tos ` / `toggle-poi ` (new mod endpoints from Phase 3) + - `files ` → `trpc.model3d.getFiles` +19. Update `.claude/skills/mod-actions/SKILL.md`'s file table (`:33-43`) and add a usage block. + +**Phase 6 — (Optional) Dedicated moderator page (~3 h)** + +20. Add `src/pages/moderator/model3ds.tsx` that lists Model3Ds by status, mirroring `src/pages/moderator/models/[id]/*` patterns if the team wants a one-stop console. **Defer** unless mods explicitly ask — the report queue + thumbnail affordance + detail-page mod bar from Phase 3 should cover most needs. + +### Gaps that need new endpoints (with proposed signatures) + +| Procedure | Signature | Purpose | +|---|---|---| +| `trpc.model3d.moderation.setNsfwLevel` | `({ id: number, nsfwLevel: number, lock?: boolean }) → Model3D` | Mod override of thumbnail-derived level; sets `lockedProperties += 'nsfwLevel'` when `lock`. | +| `trpc.model3d.moderation.toggleFlag` | `({ id: number, field: 'tosViolation' \| 'poi' \| 'minor' \| 'nsfw' \| 'unlisted' }) → Model3D` | Single endpoint per boolean; locks the field. | +| `trpc.model3d.moderation.restore` | `({ id: number }) → Model3D` | Mod-only un-delete (status: Deleted → Unpublished, clear `deletedAt`). | +| `trpc.report.createAppeal` switch case | `{ entityType: 'Model3D', entityId }` | Extend `createEntityAppealHandler` switch. | + +### Risks / open questions + +- **R6**: Two report paths (centralized `report.create` vs `model3d.reports.createForModel`) silently diverge. Centralized hits the moderator queue and runs NSFW/TOS side-effects; the direct path doesn't. Recommend Phase 2 step 10 either deprecates or hard-syncs them. +- **R7**: `getReports` `where: { [type]: { isNot: null } }` (`report.service.ts:319-321`) relies on the Prisma relation name matching the enum value. Confirmed: `Report.model3d` and `Report.model3dReview` exist (`schema.full.prisma:1334-1335`) and the enum values `model3d` / `model3dReview` (`report-helpers.ts:15-16`) match — so the `getAll` query will not need a special case. +- **R8**: `Model3D.nsfwLevel` is derived from the thumbnail's `nsfwLevel` via a batch job. If a mod overrides via `setNsfwLevel`, the next batch run will clobber it unless we honor `lockedProperties.includes('nsfwLevel')` in `updateModel3DNsfwLevels`. Confirm and add the guard in that batch job (search `updateModel3DNsfwLevels`). +- **R9**: `ReportModal.tsx`'s `useVoteForTags` is typed `entityType: 'image' | 'model'` (`:172`) — `ReportEntity.Model3D` will need either a cast/guard or an extension of `useVoteForTags`. Cheapest path: guard the call when `entityType === ReportEntity.Model3D`. +- **R10**: The `mod-actions` skill scripts auth as a moderator API key. The new `model3d` endpoints are mod-gated; verify Bearer-token tRPC calls hit `moderatorProcedure` correctly (the session shape for API-key auth must populate `ctx.user.isModerator`). Should be fine — `content.mjs` already calls `moderator.models.*` the same way. +- **R11**: NCMEC/CSAM flow: `csam.mjs` calls expect Image-shaped payloads. If a Model3D is CSAM, the thumbnail Image likely already gets reported via the existing image-CSAM path. Confirm with mods whether Model3D needs its own NCMEC path or if "report the thumbnail Image + delete the Model3D" is sufficient. Recommend the latter for v1. +- **R12**: `report.service.ts:194` NSFW side-effect for Model — `addTagVotes({ type, id, tags, ... })` — has a switch on `type` keyed to image/model. For Model3D we'd need either to add it to `addTagVotes` or skip tag-level reporting until we wire it. Skip in v1. +- **Open Q**: Should there be a "Hidden" status (between Unpublished and Deleted) for mod-forced takedowns vs creator-initiated unpublish? Today `Unpublished` is shared. Consider an audit log entry (`StrikeReason.ManualModAction` strike attached to entity) rather than a new status. +- **Open Q**: For appeals UI on Model3D detail page, model the flow off whichever image-appeal component exists. Need to grep more if scoping appeals beyond Phase 4 step 16. + +--- + +## Effort estimate + +- **M1**: **S–M** (~5-7 h) — backend count plumbing 2-3 h, profile tab page + nav 2-3 h, polish 1 h. +- **M2**: **M–L** (~10-13 h) — report queue 2 h, centralized report flow 1.5 h, mod content endpoints + UI 2-3 h, appeals + strikes 1 h, mod-actions skill 1.5 h, optional moderator page +3 h. +- **Total wall-clock**: **~15-20 hours** (without the optional dedicated moderator page); add ~3 h if Phase 6 is in scope. + +--- + +## Recommended order + +1. **M2 Phase 1** (extend `getReports` select + `getReportLink`) — unblocks the existing TODO at `pages/3d-models/[id]/[[...slug]].tsx:257`, very low risk, immediately useful to mods. +2. **M2 Phase 2** (wire `trpc.report.create` for Model3D via `ReportModal`) — finishes the report loop end-to-end. +3. **M1 Phase 1** (backend `model3dCount` cache) — required before the tab badge is meaningful and unblocks the profile tab. +4. **M1 Phases 2-4** (profile tab page + nav + polish). +5. **M2 Phase 3** (mod content endpoints + UI) — adds real action affordances beyond unpublish. +6. **M2 Phase 4** (appeals switch + strikes verification). +7. **M2 Phase 5** (`mod-actions` skill commands) — last because it leans on Phase 3's endpoints. +8. **M2 Phase 6** (dedicated `/moderator/model3ds` page) — only if mods request it after Phase 5 lands. diff --git a/docs/3d-models-implementation-tracker.md b/docs/3d-models-implementation-tracker.md new file mode 100644 index 0000000000..eb2442a965 --- /dev/null +++ b/docs/3d-models-implementation-tracker.md @@ -0,0 +1,178 @@ +# 3D Models Implementation — Live Tracker + +**Purpose**: machine-death recovery surface + parallel-agent status board for Phase 1 implementation of `docs/3d-models-plan.md` (rev 9). + +**How to use**: check this file to see what's been done, what's in flight, and what's next. Each agent commits its work to its own git worktree branch — branch names + paths are recorded below. If the machine dies, the worktree branches survive and the next session can pick up from the last recorded state. + +**Task list (live)**: use the `TaskList` tool to see runtime status. This doc is the durable record. + +--- + +## Phase 1 plan summary + +Source of truth: `docs/3d-models-plan.md` rev 9. Migration already applied to user's DB. Prisma client regenerated (49 Model3D references in `prisma/schema.prisma`). + +Workstreams (most are independent): + +| ID | Workstream | Status | Worktree branch | Owner | +|----|------------|--------|-----------------|-------| +| A | Backend foundation: `model3d.service.ts`, `model3d.router.ts`, `model3d-review.service.ts` | pending | — | — | +| B | Orchestrator PolyGen: `polygen.schema.ts` (Zod), `polyGen.handler.ts`, `generation.config.ts` registration | pending | — | — | +| C | Existing-file touch points: `ReportEntity` / `SearchIndexEntityTypes` / `commentv2.schema.ts` / `buzz.schema.ts` / `image-scan-result.ts` enum + allow-list edits | pending | — | — | +| D | UI scaffold: install three.js, build `Model3DViewer` component, scaffold `/3d-models/[id]` page stub | pending | — | — | +| E | Jobs + notifications: `updateModel3DNsfwLevels`, `updateModel3DMetrics`, comment notification SQL | pending (depends on A) | — | — | +| F | Generation form integration: add 3D Model tab to `GenerationForm.tsx`, build text-to-3D + image-to-3D sub-tabs | pending (depends on B) | — | — | +| G | "Post from Generation" + detail page + reviews modal | pending (depends on A, D) | — | — | +| H | Feature flags + mod tooling | pending | — | — | + +--- + +## Recovery protocol + +If the main session dies: + +1. **Check this doc** for the last recorded status of each workstream. +2. **`git branch | grep model3d-`** to find live agent worktree branches. +3. **`git worktree list`** to see active worktrees and their paths. +4. **`git log --all --oneline | grep -i 'Model3D\|model3d'`** for recent commits. +5. For each in-flight workstream, the worktree branch contains the agent's progress. Either: + - Resume by spawning a new agent with the same brief + worktree path + - Or merge the worktree's branch into `main`/feature branch and continue manually +6. **Run `pnpm run typecheck`** in the main worktree to spot anything broken by merged work. + +--- + +## Active agents — Wave 2 status + +Wave 2 ran, several agents died mid-flight, recovered + merged what landed: + +| Workstream | Status | Notes | +|------------|--------|-------| +| E | **DONE + merged** | 3 commits — NSFW propagation + Model3DMetric rollup + comment notifications | +| F | **DONE + merged** | Model3DGenerationForm, GenerationForm tab, QueueItem 3D branch, generate3D mutation, whatif preview | +| G (partial) | **partially merged** | Detail page rebuild + 4 components + review backend endpoints all done. **Remaining**: reviews page replacement + Post-from-Generation wiring (G2 spawned) | +| H (partial) | **partially merged** | 2 flags + surface gating done. **Remaining**: thumbnail-driven mod affordance (H2 spawned) | + +## Phase 2 — Wave 3 — COMPLETE + +All seven workstreams from `docs/3d-models-followups.md` shipped: + +| Workstream | Commit | Notes | +|------------|--------|-------| +| N: report queue surface (M2-P1) | `d0b64cd67` | `getReportsHandler` selects model3d/model3dReview joins; `getReportLink` covers both. Existing /moderator/reports tabs now produce data. | +| O: mod content endpoints + ModBar (M2-P3) | `57b53e242` | `model3d.moderation.{setNsfwLevel,toggleFlag,restore}`; `Model3DModBar` on detail page (mod-only); `updateModel3DNsfwLevels` honors `lockedProperties`. | +| P: count plumbing (M1-P1) | `4a7e7a51d` | `userModel3DCount{,Sfw,Public}Cache` + `UserContentOverview.model3dCount`; refresh on publish/unpublish/delete. | +| Q: centralized ReportModal (M2-P2) | `5751b9dc3` | ReportModal accepts Model3D + Model3DReview across NSFW/TOS/AdminAttention/Spam; detail-page report button wired; NSFW side-effect flips `Model3D.nsfw`. | +| R: profile tab + nav (M1-P2..4) | `6e89a721f` | New `/user/[username]/3d-models` page + ProfileNavigation entry behind `model3dFeed`. ProfileLayout2 regex tightened for hyphenated subpages. | +| S: appeals switch + strikes verify (M2-P4) | `505f4c5a0` | `createEntityAppealHandler` accepts `EntityType.Model3D`; detail page surfaces appeal CTA when isOwner sees Unpublished/Deleted. Strikes confirmed working as-is. | +| T: mod-actions skill (M2-P5) | `5da10170e` | `.claude/skills/mod-actions/model3ds.mjs` + SKILL.md updates. 12 commands; strikes deferred to existing strikes.mjs. | + +**Agent reliability note**: Wave 2 agents (Q/R/S) and Wave 3 (T) were all blocked by Edit/Write denials in their isolated worktrees. Their research was solid and complete; landed inline based on their plans. Wave 1 agents (N/O/P) shipped cleanly via worktrees as intended. + +Plan source of truth: `docs/3d-models-followups.md` (rev 1; profile feed + moderation phases). + +## Phase 2 — Wave 4 (post-launch polish) — COMPLETE + +User-driven follow-ups from card/mod review + generator surfacing: + +| Workstream | Commit | Notes | +|------------|--------|-------| +| U: mod actions → single dropdown menu | `7ad54c7ca` | `Model3DModBar` (button-row + inline Menu + Popover) replaced with `Model3DModMenu` — canonical Mantine pattern (LegacyActionIcon + IconDotsVertical + single Menu.Dropdown). Destructive confirms via `openConfirmModal`; NSFW level edits via a Mantine Modal. | +| V: card redesign + inline preview | `071e06b6a` | `Model3DCard` rebuilt with ModelCard-shape footer (UserAvatarSimple + stat/rating chips). NSFW handling via `ImageGuard2` (added `'model3d'` to `ConnectType` union). Header IconEye Preview button lazily loads primary file via `trpc.model3d.getFiles` and renders three.js viewer inline as an absolute overlay. | +| W: V2 generator integration | `07377c88b` | Registers PolyGen ecosystem (`ECO.PolyGen=71`, `BM.PolyGen=90`) and `'3D Models'` category in `GenerationFormV2`. Extends `WorkflowCategory`/`OutputType`/`MediaType` unions to include `'model3d'`. Adds `txt2model3d` + `img2model3d` workflow configs (feature-flagged on `model3dGenerator`, `noSubmit: true`). `Model3DGenerationForm` is rendered as the workflow body inside `GenerationForm.tsx` — bypasses the unified `generateFromGraph` path and uses the existing `generate3D` + `generate3DWhatIf` mutations. Empty `polygen-graph.ts` placeholder is registered in `ecosystem-graph.ts`. | + +**Reverted along the way**: `aaa9b7541` — first W attempt edited the legacy `GenerationForm.tsx` (no end-user importers; `GenerationFormLegacy` is dead code). Reverted via `96f48ade1`; replaced with the V2 integration above. + +**Open follow-ups from W** (intentional, not blockers): +- `BaseModelRecord.type` for PolyGen is `'image'` because Prisma `MediaType` enum has no `'model3d'` variant. The record is `hidden: true` so it never surfaces in pickers. Future migration could extend the enum. +- `WORKFLOW_TAGS` has no `'model3d'` tag. Orchestration would mis-tag PolyGen submissions as `'vid'`, but the dispatcher in `createEcosystemStep` throws before reaching that code (PolyGen submits via `generate3D`, not `generateFromGraph`). Add the tag if the unified path is ever wired. +- The empty `polyGenGraph` means PolyGen contributes zero form nodes through the unified graph (intentional — the standalone form IS the workflow body). If a future workstream wants graph-driven PolyGen inputs, build out the nodes + a handler entry in `createEcosystemStep`. + + +## Active agents — Wave 2.5 (continuation) — DONE + +| Workstream | Status | Notes | +|------------|--------|-------| +| G2: reviews page + post-from-gen | **DONE + merged** | 3 commits — reviews page (`6cc97db7a`), Post-from-Gen wiring + `getByWorkflowId` (`53a826201`), publish-hook flips Model3D Draft→Published (`8a9b1cf04`). Merged with import conflict resolved (H2's `getByThumbnailImageId` + G2's `getByWorkflowId` coexist). | +| H2: mod tooling | **DONE + merged** | 3 commits — `getByThumbnailImageId` procedure + `Model3DModAction` component + wired into `src/pages/moderator/images.tsx:655`. | + +## ✅ Phase 1 complete + +All 8 workstreams (A–H) merged on `main`. `pnpm run typecheck` clean across all Model3D code. + +Phase 1 surface summary: +- Schema + migration applied +- Services: model3d / model3d-review / model3d-report +- Router: model3d (with `reviews` and `reports` sub-routers) + 12 procedures +- Orchestrator: PolyGen handler + Zod schema + generation-config registration +- UI: 3D Model generation form, queue card branch, detail page, reviews page, reviews modal +- Viewer: three.js + GLTFLoader (dynamic-imported) +- Jobs: NSFW propagation + Model3DMetric rollup + comment notifications +- Mod affordance: thumbnail-driven "Also unpublish parent Model3D" +- Feature flags: `model3d-feed` + `model3d-generator` (Flipt, mod-only at launch) +- Post-publish hook: linked Model3D auto-flips Draft → Published + +Open follow-ups (intentionally deferred per plan): +- ClickHouse download event emission (Model3DMetric.downloadCount currently stays 0) +- Post-edit page surfacing Model3D-specific fields (currently passes `?model3dId=` through but doesn't render the form) +- Reviews pagination switched from page-based to cursor-based if scale demands it +- Dedicated `model3d` Meilisearch index implementation (search-parser is registered but routes nowhere yet) +- User uploads (Phase 3, schema is upload-ready) + +## Active agents — Wave 1 (done, merged) + +See "Commit log" below. + +## Integration plan (once C lands) + +Merge order: **D → B → C → A** (least conflict risk; A touches the most, lands last). + +1. `cd /home/luis_rojas/Work/civitai` +2. `git merge --no-ff worktree-agent-a7e6b0ab4d05237b9` (D) +3. `pnpm install` (picks up three.js) +4. `git merge --no-ff worktree-agent-a240a7c10f27a2a52` (B) +5. `git merge --no-ff worktree-agent-ad3d34f1b06bd20c6` (C — once committed) +6. `git merge --no-ff worktree-agent-a032ad75027b491bc` (A) +7. Reconcile B's `upsertModel3DDraft` TODO → A's actual `upsertModel3D` shape (one-line edit in `polyGen.handler.ts`) +8. `rm src/pages/3d-models/[id]/.gitkeep` +9. `pnpm run typecheck` (expect 0 new errors; pre-existing main errors unchanged) +10. `pnpm run lint` +11. `git commit -am "feat(model3d): integrate Phase 1 wave 1 (workstreams A, B, C, D)"` +12. `git worktree remove` each worktree (cleanup) + +## Monitoring commands + +```bash +# Live task status (runtime, in-conversation) +# Use the TaskList tool in the assistant session. + +# Worktrees + branches survive across machine restarts: +git worktree list +git branch | grep -i model3d + +# Recent commits across all branches: +git log --all --oneline --since="1 hour ago" | head -30 +``` + +--- + +## Commit log (post-merge) + +| Date | Workstream | Branch merged | Notes | +|------|------------|---------------|-------| +| 2026-05-27 | groundwork | (main) `2f7b86a7a` | schema + migration + docs + civitai-client bump | +| 2026-05-27 | D | `worktree-agent-a7e6b0ab4d05237b9` @ `9efe10370` | three.js + viewer + 3 page stubs | +| 2026-05-27 | B | `worktree-agent-a240a7c10f27a2a52` @ `8ff736a52` | PolyGen schema + handler + generation.config registration | +| 2026-05-27 | C | `worktree-agent-ad3d34f1b06bd20c6` @ `dc27eb04d` | 10 touch-point files (enums, allow-lists, collection.utils, job-queue, user.service) | +| 2026-05-27 | A | `worktree-agent-a032ad75027b491bc` @ `5c60f5645` | services + router + Zod + router registration | +| 2026-05-27 | reconcile A↔B | (main) `4e8570350` | added upsertModel3DFromWorkflow; wired polyGen.handler.ts; `currencies: []` for WorkflowTemplate | + +**Phase 1 Wave 1 complete. `pnpm run typecheck` passes.** + +--- + +## Next steps after Phase 1 lands + +- Apply the migration to the next environment (staging/prod) per user request. +- Phase 2: feed broadening, profile tab, community "Makes/Uses" Post linkage. +- Phase 3: user uploads (deferred; schema is upload-ready). diff --git a/docs/3d-models-plan.md b/docs/3d-models-plan.md new file mode 100644 index 0000000000..6c77db9686 --- /dev/null +++ b/docs/3d-models-plan.md @@ -0,0 +1,433 @@ +# 3D Models Support — Implementation Plan + +**Status**: Rev 9 — open questions resolved, ready to implement. +**Author**: AI assistant, with Luis +**Date**: 2026-05-27 (rev 9) +**Project framing**: hackathon pilot / tech demo. v1 ingests 3D models from the orchestrator's PolyGen recipe (Meshy via Fal). User uploads are explicitly OUT of v1 to dodge the moderation/scanning lift, but the data model is upload-ready for future use. + +--- + +## 1. Executive Summary + +We're adding a first-class 3D Model content type to Civitai, populated by **AI generation** (PolyGen / Meshy text-to-3D and image-to-3D, via the orchestrator). Users browse a feed, view a 3D model in-browser, see generation details, react to thumbnails (not to the model), discuss in comments, review with stars, and post "Makes/Uses" (e.g. how they used the model in a game). + +**Locked architectural calls**: + +- **New top-level entity** (`Model3D` + `Model3DFile`). No versioning. A `Model3D` carries its thumbnail, license, and 1..N files (one per format: GLB primary + FBX/OBJ/USDZ alternates). +- **Source = generation, not upload**, in v1. Schema is upload-ready (`workflowId` nullable) so the future upload flow is a code-only change. +- **Generation via `invokePolyGenStepTemplate`** (civitai-client PolyGen). Two operations: `textTo3D`, `imageTo3D`. Engine `fal`, model `meshy`. +- **No `Model3DReaction` table**. Reactions ride on the thumbnail Image (which is itself an `Image` row, reusing `ImageReaction`). +- **No `Model3DDownloadHistory` table**. Download events go to ClickHouse; the per-model rollup is denormalized into `Model3DMetric.downloadCount`. +- **Reviews in scope** — `Model3DReview` parallel to `ResourceReview`, with the `/3d-models/[id]/reviews` route. +- **Thumbnail comes from the generator** (PolyGenOutput.thumbnail). Only the thumbnail Image gets NSFW/CSAM scanned for v1. +- **Routes**: `/3d-models`, `/3d-models/[id]`, `/3d-models/[id]/reviews`. +- **New `Model3DLicense` table** tailored for 3D-asset licensing (printing + games + viz). + +**Effort**: Phase 1 (schema + generation panel + save-as-Model3D + detail page + reviews) **L** — honest. Phase 2 (feed + community Posts + makes/uses surfaces) S–M. Phase 3 (user uploads) deferred. + +--- + +## 2. Architectural Decisions + +### 2.1 New top-level entity (`Model3D`), no versioning, upload-ready + +A printable / game-ready / visualization 3D asset has nothing to do with what Civitai calls a "Model" (an AI inference resource). Separate entity, no `ModelType` extension. + +**No versioning** in v1. A `Model3D` is the unit; if someone iterates, that's a new `Model3D`. (Future: easy to add `Model3DVersion` later if real demand emerges.) + +**Upload-ready**: `workflowId` and `sourceImageId` are nullable. Today every row will have them set (generation provenance). Tomorrow's user-upload flow writes rows with both NULL. No schema migration needed for that pivot. + +### 2.2 Source = orchestrator PolyGen (v1 only) + +Integration uses the **async workflow pattern**, not the synchronous template-eval endpoint: + +- Submit via `submitWorkflow` with a `PolyGenStep` (`$type: 'polyGen'`) — mirrors `sora.handler.ts` / `wan` exactly. Result streams back via the existing workflow result handler; the user sees a generation card in their queue with status / retry / cost. +- **Do not** call `invokePolyGenStepTemplate` directly (synchronous, bypasses queue/billing/retries). + +Two operations on Meshy: + +- `MeshyTextTo3dFalPolyGenInput` — prompt → 3D +- `MeshyImageTo3dFalPolyGenInput` — source image URL → 3D + +Meshy params surfaced in the form (all from `MeshyFalPolyGenInput`): `targetPolycount` (100–300k, default 30k), `topology` (`quad`|`triangle`), `symmetryMode` (`off`|`auto`|`on`), `shouldRemesh`, `enablePbr`, `texturePrompt`, `enableRigging`, `enableAnimation`, `seed`. Text-to-3D adds `prompt`, `mode` (`preview`|`full`), `enablePromptExpansion`. Image-to-3D adds `imageUrl`, `shouldTexture`. + +`PolyGenOutput` returns: `model: Model3dBlob` (primary, e.g. `format='glb'`), `fbxModel?: Model3dBlob` (optional FBX), `thumbnail?: ImageBlob`. The result handler: + +1. Ingests the source image (image-to-3D only) as an `Image` row first, mirroring Sora's `sourceImageSchema` pattern, before the workflow submit. The resulting `Image.url` is what goes into `imageUrl`. `Model3D.sourceImageId` references this row. +2. On result, copies returned blobs (`model.url`, `fbxModel?.url`, `thumbnail?.url`) into our S3 (`3d/` prefix). `Model3dBlob.url` is nullable and may need a re-presign — retry on stale URL. +3. Snapshots the full PolyGenInput into `Model3D.generationParams` (Json) — the Generation Details panel reads from here. +4. Creates `Model3DFile` rows (one per format, GLB marked `isPrimary`). + +`Model3dBlob.format` is free-string by contract; the result handler normalizes (`toLowerCase().replace(/^\./, '')`) before insert so `"GLB"` / `".glb"` / `"glb"` all hash to the same row. + +**Client status**: `@civitai/client@0.2.0-beta.67` (currently installed) exports `submitWorkflow`, `PolyGenStep`, `PolyGenStepTemplate`, `FalPolyGenInput`, `MeshyFalPolyGenInput`, `MeshyTextTo3dFalPolyGenInput`, `MeshyImageTo3dFalPolyGenInput`, `Model3dBlob`. **Unblocked.** + +### 2.3 Thumbnail comes from the generator + +`PolyGenOutput.thumbnail` is an `ImageBlob`. We save it as a regular `Image` row, scan it through the standard NSFW/CSAM pipeline, then link `Model3D.thumbnailImageId = image.id`. + +If `thumbnail` is missing from the output, we fall back to `sourceImage` (for image-to-3D) or a placeholder render. The detail page must always have *something* to show. + +The thumbnail Image lives in a Post (per the `Image.postId` FK constraint). The "Post from Generation" CTA creates that Post when the user saves the generation; community Posts ("I used this in my game") are separate Posts linked via `Post.model3dId`. + +### 2.4 Viewer → three.js GLB renderer (new component, dynamic-imported) + +Primary format is GLB (Meshy default). `three.js` + `GLTFLoader` + `OrbitControls` handles GLB natively, including PBR materials and rigging. Dynamic-imported (`next/dynamic`, `ssr: false`) so the bundle hit is paid only by viewers. + +**This is a from-scratch component**, not a rebrand of anything that exists. `three` is not currently in `package.json` — Phase 1 needs to install `three` + `@types/three` and verify bundle delta with `next build`. + +**Queue card preview**: do NOT spin up a WebGL context per queue card (5 queued generations = 5 WebGL contexts on the page = bad). Queue cards show the thumbnail Image only; the full viewer instantiates on the detail page click. + +Other formats (FBX, OBJ, USDZ, STL) are stored alongside but the in-browser viewer only renders GLB. The "Files" dropdown lets users download any format. + +### 2.5 Files → multiple formats per Model3D, dropdown selector + +`Model3DFile` rows are 1..N per Model3D, one per `format`. Unique on `(model3dId, format)`. An `isPrimary` boolean marks the default viewer/download format (typically GLB). The detail page renders a single dropdown "Format: GLB / FBX / OBJ" with the primary pre-selected. + +**No file size cap in v1.** Storage is not a constraint right now per product direction. Realistically Meshy outputs are 5–50 MB so caps would rarely bite anyway. Revisit if egress costs spike post-launch. + +### 2.6 Community Posts ("Makes/Uses") → `Post.model3dId` + +Community members create Posts to show how they used a Model3D (e.g. screenshots from a game, renders from Blender). These Posts get `Post.model3dId = .id`. The detail page renders a "Makes & Uses" rail showing these Posts. + +The creator's auto-Post (containing the generation thumbnail) and community Posts both live in the `Post` table, differentiated by `userId`. + +### 2.7 Licensing → new `Model3DLicense` table + +Same as rev 4: separate `Model3DLicense` table with print-farm / derivatives / redistribution flags. Generated content defaults to a "Civitai Generated" license (configurable). Users can change it at "Post from Generation" time. + +Seeded templates: CC-BY 4.0, CC-BY-NC 4.0, Personal Use Only, No Commercial Print Farm, All Rights Reserved, Custom. + +### 2.8 Tag taxonomy → generic 3D-asset tags + +Not print-specific. Seeds: + +- **Subject** (categories): character, creature, environment, prop, vehicle, architecture, furniture +- **Style** (filters): low-poly, stylized, realistic, abstract, sci-fi, fantasy + +Print-specific tags (FDM, Resin, Supports Required, Print-in-Place, etc.) are dropped — the goal is to be 3D-generic, not print-focused. + +### 2.9 Discovery / search → dedicated `model3d` Meilisearch index + +A new index. Lots of wiring (new instantsearch tab, new parser, separate reindex cron) but clean separation. + +### 2.10 NSFW scanning + mod tooling → thumbnail Image is the single signal + +The 3D model file itself is NOT scanned for content (mod team has no tooling for 3D content moderation). We rely on: + +- **Generator-provided thumbnail** flowing through the standard `Image` NSFW + CSAM scan pipeline. +- **PolyGen `allowMatureContent` query param** to gate generation by user moderation level. +- **Mod team review queue** for reports. + +**Mod tooling — basic, thumbnail-driven**: when a moderator actions a thumbnail Image (block, NSFW level change, delete), they get a "Also action the parent Model3D" affordance on the existing image-mod page. Action propagates: delete thumbnail → Model3D goes to `Unpublished` (or `Deleted` if mod chooses); NSFW level change → propagated by the `updateModel3DNsfwLevels` batch job (§4). No 3D-specific mod queue in v1. + +Future user uploads will need real 3D content moderation; that's a Phase 3 problem. + +### 2.11 Entry point + feature flags → split feed / generator flags + +**Two Flipt flags, both mod-only at launch**: + +- `model3d-feed` — gates *viewing*: feed page, detail page, comments, reviews, profile tab. If you have feed access you can browse existing Model3Ds, comment, react, review — but you cannot create. +- `model3d-generator` — gates *creating*: the new "3D Model" segmented option in the generation panel. Implies `model3d-feed` (no point in generation if you can't see results). + +Both flipt-config'd; mod-only to start, opened to broader audiences when QA + content seeding catches up. + +The generation panel today (`src/components/ImageGeneration/GenerationForm/GenerationForm.tsx`) has segmented controls for Image and Video. With `model3d-generator` on, we add a third: **3D Model**. The form has two sub-tabs: text-to-3D and image-to-3D. + +Result handling follows the established pattern (`GeneratedImageActions.tsx`): + +1. Workflow completes; queue card shows the generation's **thumbnail Image** (not an inline viewer — see §2.4). +2. User clicks "Post from Generation" → mutation creates an empty `Post`, redirects to `/posts/[postId]/edit`. +3. On the edit page, the user fills in `Model3D` metadata (name, description, tags, license) — same edit page hosts both Post fields and Model3D fields, since they're created together. +4. On publish: `Model3D.status = Published`, `Post` flipped to public, `Post.model3dId` set. + +The Model3D row itself is created at the time the workflow result handler runs (server-side, on completion), in `Draft` status, tagged with `workflowId`. `workflowId` is UNIQUE — re-submitting the "Post from Generation" CTA returns the existing draft instead of creating a duplicate. + +### 2.12 Reviews → modal with image attachments via a Post + +The write-review surface is a modal (mirroring the existing `EditResourceReviewModal` for AI models). **The modal supports image attachments** — users reviewing a 3D model can post photos/renders of how they used it (e.g. "here's how this character looks in my game"). This is product-prioritized for Model3D where the AI-side connection is loose. + +Implementation: the review modal optionally creates a `Post` (containing the attached images) and links it via `Post.model3dReviewId` (new nullable `@unique` column). The review detail surfaces those images inline. + +A `Model3DReview` can have at most one associated Post (`Post.model3dReviewId @unique`). Empty-images reviews skip Post creation entirely. + +### 2.13 Generation cost preview → orchestrator `whatif` + +PolyGen's `whatif` query param (existing orchestrator pattern) returns the Buzz cost based on input params. The form calls `whatif` on param-change (debounced) to show estimated cost inline. No separate billing infrastructure needed — pricing lives in the orchestrator. + +### 2.14 Routes + +| Route | View | +| -------------------------------- | --------------------------------------------------------------------------------- | +| `/3d-models` | Feed (Meilisearch-backed) | +| `/3d-models/[id]` | Detail: preview, general info, files dropdown, generation details, comments, makes/uses | +| `/3d-models/[id]/reviews` | Reviews list + write-review CTA | +| `/user/[username]/3d-models` | Per-user profile tab | +| (generation surface) | Existing generation panel, new "3D Model" type | + +--- + +## 3. Schema Changes + +All additive. Per CLAUDE.md, we write the SQL and surface it for manual application — no `prisma migrate deploy`. + +Migration file: `prisma/migrations/20260526120000_add_3d_models/migration.sql`. + +### New tables + +```prisma +model Model3D { + id Int @id @default(autoincrement()) + name String @db.Citext + description String? + userId Int + thumbnailImageId Int? @unique // nullable + SetNull; required-at-publish enforced in app + licenseId Int + licenseDetails String? + workflowId String? @unique // orchestrator workflow ID; UNIQUE prevents dup Post-from-Generation + sourceImageId Int? // image-to-3D source + generationParams Json? // PolyGen input snapshot + status Model3DStatus @default(Draft) + nsfw Boolean @default(false) + tosViolation Boolean @default(false) + poi Boolean @default(false) + minor Boolean @default(false) + unlisted Boolean @default(false) + lockedProperties String[] @default([]) + availability Availability @default(Public) + nsfwLevel Int @default(0) + meta Json @default("{}") + // timestamps + deletion fields. No `scannedAt` — only the thumbnail Image is scanned in v1. +} + +model Model3DFile { + id Int @id @default(autoincrement()) + model3dId Int + name String + url String + sizeKB Float // no cap in v1 + format String // normalized lowercase: 'glb' | 'fbx' | 'obj' | 'usdz' | 'stl' | ... + isPrimary Boolean @default(false) // at most one per Model3D + // virusScanResult defaults to 'Success' for v1 (orchestrator-trusted); set 'Pending' when uploads land + @@unique([model3dId, format]) +} + +model Model3DLicense { /* CC-BY, Personal Use Only, etc. */ } +model Model3DReport { /* per-entity report */ } +model Model3DReview { /* rating 1..5 + recommended + details, unique per (model3dId, userId) */ } +model Model3DReviewReport { /* report-on-review */ } +model TagsOnModel3D { model3dId, tagId } +model Model3DEngagement { /* Favorite / Hide / Notify */ } +model Model3DMetric { /* downloadCount sourced from ClickHouse, ratingAvg, etc. */ } + +enum Model3DStatus { Draft Published Unpublished Deleted } +enum Model3DEngagementType { Favorite Hide Notify } +// No Model3DFileType enum — `format` is free-text String to match Meshy output +``` + +### Removed from earlier revs + +- **`Model3DReaction`** — react on the thumbnail Image instead. Saves a table + service + UI plumbing. +- **`Model3DDownloadHistory`** — download events go to ClickHouse. Aggregate into `Model3DMetric.downloadCount`. +- **`Model3DFileType` enum** — `format` is String now. + +### Existing-table touch list + +| Surface | Change | Notes | +|---|---|---| +| `Thread` | `model3dId Int?`, `model3dReviewId Int?` | both `@unique`, `SetNull`. Reviews get comment threads too. | +| `Post` | `model3dId Int?`, `model3dReviewId Int?` | `model3dId` for community Posts + creator generation Post; `model3dReviewId @unique` for review-with-images Posts | +| `BuzzTip` schema validation | extend allow-list enum in `src/server/schema/buzz.schema.ts:136` | DB is polymorphic, schema isn't | +| `Collection.CollectionType` enum | add `Model3D` | hardcoded enum | +| `CollectionItem` | add `model3dId Int?` + extend unique constraint | four-FK pattern | +| `EntityType` enum | add `Model3D` | for `JobQueue` background pipelines | +| `CosmeticEntity` enum | add `Model3D` | optional | +| `TagTarget` enum | add `Model3D` | + `image-scan-result.ts:646` default-target list | +| `ReportEntity` enum (`src/shared/utils/report-helpers.ts`) | add `Model3D`, `Model3DReview` | hardcoded | +| `commentv2.schema.ts` | add `Model3D` + `Model3DReview` to enum lists | two locations | +| Notifications | new SQL for `new-3d-model-comment` + `new-3d-model-review` | hand-written | +| `ProfileNavigation.tsx` | add 3D Models tab | hardcoded static list | +| `SearchIndexEntityTypes` (`src/components/Search/parsers/base.ts:25-34`) | add `'Model3D'` key (PascalCase) + new parser | for new Meilisearch index | +| `GenerationForm.tsx` segmented control | add "3D Model" alongside existing Image / Video | (no Audio tab today — earlier rev had this wrong) | +| `nsfwLevels.service.ts` | add `updateModel3DNsfwLevels` batched job | propagates thumbnail Image's nsfwLevel up to Model3D | +| Metrics rollup job | new `updateModel3DMetrics` | populates Model3DMetric from comments/reviews/collections/thumbnail ImageMetric reactions | +| `Image.metadata` | (no schema change) add `kind: 'render' \| 'photo'` | UI-only | + +### Indexes + +- `Model3D (userId, status, publishedAt DESC)` — profile pages +- `Model3D (status, publishedAt DESC)` — feed +- `Model3D (status, nsfwLevel, publishedAt DESC)` — NSFW-aware feed +- `Model3D (workflowId hash)` — find Model3D by orchestrator workflow +- `Model3D (sourceImageId hash)` — find Model3Ds derived from an image +- `Model3DFile (model3dId hash)`, `(model3dId, format)` unique +- `Model3DReview (model3dId, userId)` unique +- All other report/engagement/metric indexes match existing patterns. + +### Seed data + +- `Model3DLicense` rows: CC-BY 4.0, CC-BY-NC 4.0, Personal Use Only, No Commercial Print Farm, All Rights Reserved, Custom. +- `Tag` rows with `target = Model3D`: character, creature, environment, prop, vehicle, architecture, furniture, low-poly, stylized, realistic, abstract, sci-fi, fantasy. `ON CONFLICT` handling for tags that already exist. + +--- + +## 4. Phased Plan + +### Phase 1 — Schema + generation panel + save-as-Model3D + detail page (M) + +Behind feature flags `model3d-feed` + `model3d-generator` (both mod-only at launch — see §2.11). + +**Backend** + +- Schema migration: new tables + existing-table touch list (§3) + seed data. +- New service `src/server/services/model3d.service.ts`: `upsertModel3D`, `getModel3DById`, `getModel3DsInfinite` (mod-only), `publishModel3D`, `unpublishModel3D`, `deleteModel3D`, `getModel3DFiles` (signed download URLs). +- New router `src/server/routers/model3d.router.ts`. +- **Orchestrator integration** (unblocked — `@civitai/client@0.2.0-beta.67` ships PolyGen): + - New `src/server/services/orchestrator/ecosystems/polyGen.handler.ts` mirroring `sora.handler.ts` shape. Builds a `PolyGenStep` for `submitWorkflow`. + - New `src/server/orchestrator/polygen/polygen.schema.ts` — discriminated union mirroring `MeshyTextTo3dFalPolyGenInput` / `MeshyImageTo3dFalPolyGenInput` plus a `sourceImageSchema` for image-to-3D (mirroring Sora's source-image ingestion). + - Register in `src/server/orchestrator/generation/generation.config.ts` (sibling to existing video/image configs). + - **Workflow result handler** (server-side, runs on workflow completion): ingests `PolyGenOutput.thumbnail` as an `Image` row via the existing image-ingest pipeline (NSFW + CSAM scan); copies `model.url` and `fbxModel?.url` blobs into our S3 (`3d/` prefix); normalizes `Model3dBlob.format` (lowercase, strip leading dot); creates the `Model3D` row in `Draft` with `workflowId` set; creates `Model3DFile` rows (one per format). + - Idempotence: `Model3D.workflowId` is UNIQUE; re-running the handler on the same workflow returns the existing draft. +- `commentsv2.service.ts` + `commentv2.schema.ts` enum edits. +- `BuzzTip` schema validation edit. +- `ReportEntity` enum edit (`src/shared/utils/report-helpers.ts`). +- New `Model3DReport` + `Model3DReviewReport` tables + report router edits. +- Mod-queue UI edit: `/moderator/reports` accepts the new report tables. +- New comment notifications: `new-3d-model-comment`, `new-3d-model-comment-response`, `new-3d-model-comment-nested`. +- `EntityType` + `TagTarget` enum additions; `image-scan-result.ts:646` edit. +- **New batch jobs**: + - `updateModel3DNsfwLevels` in `src/server/services/nsfwLevels.service.ts` — propagates `Image.nsfwLevel` (thumbnail) to `Model3D.nsfwLevel`. Without this, the column stays at 0 forever. + - `updateModel3DMetrics` — populates `Model3DMetric` from `CommentV2`, `Model3DReview`, `CollectionItem`, `BuzzTip`, and the thumbnail Image's `ImageMetric` (for `reactionCount`). + +**Frontend** + +- **Generation form**: new "3D Model" segmented-control option in `GenerationForm.tsx` (alongside existing Image and Video — there is no Audio today). Two sub-tabs: text-to-3D, image-to-3D. Form fields mirror PolyGen schema (all of `MeshyFalPolyGenInput` + the operation-specific fields). +- **Generation queue card**: new 3D-model card showing the thumbnail Image (NOT an inline WebGL viewer — multiple cards on the page would create N WebGL contexts). "Post from Generation" CTA mirrors the existing image/video flow: creates empty Post → redirects to `/posts/[id]/edit`. The Model3D draft was already created by the workflow result handler; the edit page binds them together via `Post.model3dId`. +- **Detail page** `/3d-models/[id]/[[...slug]].tsx`: + - 3D viewer (`` rebranded ``, GLB-first via three.js + GLTFLoader) + - General info (name, description, creator, license) + - Files dropdown (select format → download) + - Generation Details section (prompt, topology, polycount, seed, source image if image-to-3D) + - Comments section (existing Thread/CommentV2) + - "Makes & Uses" rail (community Posts where `Post.model3dId = id`) +- **Reviews page** `/3d-models/[id]/reviews.tsx` — list + write-review CTA. Star rating, recommend checkbox, free-text details. + +**Critical files** + +- `prisma/schema.prisma` + migration +- `src/server/services/model3d.service.ts` + `model3d-review.service.ts` + `model3d-report.service.ts` +- `src/server/routers/model3d.router.ts` +- `src/server/orchestrator/polygen/polygen.schema.ts` + workflow handler +- `src/server/notifications/comment.notifications.ts` (large edit) +- `src/components/Model3D/Viewer/Model3DViewer.tsx` (three.js + GLTFLoader, dynamic-imported) +- `src/components/Generation/Model3D/Model3DGenerationForm.tsx` +- `src/pages/3d-models/[id]/[[...slug]].tsx`, `[id]/reviews.tsx` +- Generation panel content-type selector +- `src/server/services/feature-flags.service.ts` + +### Phase 2 — Discovery (feed + profile tab + search bar entry) + community Posts (S–M) + +Broadens the audience of `model3d-feed` / `model3d-generator` from `mod` to wider groups via Flipt (no code change). + +- New `src/server/search-index/model3d.search-index.ts` — dedicated Meilisearch index. +- `SearchIndexEntityTypes` in `src/components/Search/parsers/base.ts:25-34` — add `model3d` + new parser + new instantsearch tab. +- New `src/pages/3d-models/index.tsx` — feed grid. +- New `src/components/Cards/Model3DCard.tsx`. +- New `src/pages/user/[username]/3d-models.tsx` — profile tab. +- `ProfileNavigation.tsx` static-list edit + `userProfile.overview` count. +- `PostUpsertForm2` — allow linking a Post to a `Model3D` via `model3dId`. Add `Image.metadata.kind = 'photo'` selector for "Makes/Uses" posts. + +### Phase 3 — User uploads (deferred) + +- New upload flow analogous to existing model uploads, but with content moderation gating. +- Writes rows with `workflowId = NULL` (schema already supports this). +- Requires: 3D content moderation strategy, virus/integrity scanning beyond pass-through, mod tooling for 3D files. + +--- + +## 5. Content Policy + +Same as rev 4 — generated content is gated by the orchestrator's existing user-tier policies (`allowMatureContent` query param). Mod queue applies to user-uploaded content via `Model3DReport`. + +- **NSFW / printed photos** in "Makes/Uses" Posts: standard `Image` moderation pipeline. +- **Weapons / firearms**: total ban; orchestrator-side prompt filtering + mod review. +- **POI / real-person likenesses**: standard Civitai POI rules apply to the thumbnail Image and the prompt. +- **Copyrighted IP**: report-driven via `Model3DReport`. New "Copyrighted IP" report reason. + +The orchestrator-side `allowMatureContent` toggle is the primary lever for v1 — users who can't see mature content can't generate it. + +--- + +## 6. Risks + +1. **Viewer perf on FBX-with-rigging** — `enableRigging` + `enableAnimation` Meshy outputs may exercise three.js paths we haven't tested. Soft warning above some triangle count threshold. +2. **NSFW moderation surface** — only the thumbnail Image gets scanned. Prompts that produce questionable 3D content but innocuous thumbnails slip through. Mitigation: `allowMatureContent` gating + report queue. +3. **Reaction-on-thumbnail UX leak** — accepted trade-off, but with side effects to call out: (a) user's profile "liked content" surfaces liked thumbnail Images, not Model3Ds — needs a query shaper to lift these into "liked 3D models" or remain as Images; (b) replacing a thumbnail (changing `Model3D.thumbnailImageId`) drops the reaction count visible on the card unless the old `Image` row is preserved; (c) feed sort-by-popular reads `Model3DMetric.reactionCount` (denormalized rollup from the thumbnail's `ImageMetric`) — needs the rollup job to run. Mitigations are real but each costs work. +4. **PolyGen cost** — Meshy charges per generation; need a Buzz pricing model. Use PolyGen's `whatif` query param to preview cost at form-load. Coordinate with billing. +5. **Generation reliability** — Meshy generation can fail or time out. Standard orchestrator failure handling applies, but the UX needs a clear "Generation failed, try again" path. +6. **`Model3dBlob.url` is nullable / may expire** — workflow result handler must handle the "blob not yet available" and "URL expired" cases. Standard re-presign retry pattern. +7. **ClickHouse downloads are lossy** — fire-and-forget on download click; if ClickHouse is unavailable we lose the event. Acceptable for counts; flagged for visibility. +8. **Notifications copy-paste burden** — new notification types per `comment.notifications.ts` pattern. Budget 4–6. +9. **Schema enumeration sites** — 15+ surfaces touched (see §3 touch list, now including `ReportEntity`, `SearchIndexEntityTypes`, NSFW + metric rollup jobs). Easy to miss; grep audit before merge. + +--- + +## 7. Out of Scope (v1) + +- **User uploads** — Phase 3. Schema is upload-ready (nullable `workflowId`), code path is not. +- **GLB derivation from STL** — irrelevant; we receive GLB directly from Meshy. +- **OBJ / GLTF / 3MF / STEP upload paths** — Phase 3. +- **Slicer integration / G-code / print-time estimates**. +- **Marketplace / paid downloads**. +- **Print-farm verification / "verified printer" badges**. +- **Multi-part assembly UI**. +- **Vault eligibility for 3D files**. +- **Bulk-import from Printables / Thingiverse / MakerWorld**. +- **Versioning** — `Model3D` is atomic; iteration = new Model3D. + +--- + +## 8. Decision Log + +| # | Question | Decision | Notes | +|---|---|---|---| +| 6.1 | Post → Model3D linkage | Nullable `model3dId` column on `Post` | Mirrors `modelVersionId` | +| 6.2 | License model | New `Model3DLicense` table | 3D-asset-specific dimensions | +| 6.3 | File size cap | **No cap in v1** | Storage not a constraint per product direction; revisit if egress spikes | +| 6.4 | Viewer cap for huge files | Soft warning above threshold | Still render | +| 6.5 | Tag taxonomy | Generic 3D-asset tags (subject + style) | Dropped print-specific (FDM, Resin, etc.) | +| 6.6 | "Create" entry point | Top-nav Generate menu, new "3D Model" type | Replaces upload-wizard idea | +| 6.7 | Reviews | **IN scope (v1)** | `/3d-models/[id]/reviews`, parallel to ResourceReview | +| 6.8 | Weapons / firearms policy | Total ban | Orchestrator prompt filter + mod review | +| 6.9 | POI policy | Standard Civitai POI rules apply to thumbnail + prompt | | +| 6.10 | Meilisearch | Dedicated `model3d` index | New entity ⇒ new index | +| 6.11 | GLB derivation | N/A — receive GLB directly from Meshy | | +| 6.12 | New entity vs. extend `Model` | New entity (locked) | "Model" in Civitai means AI generation resource | +| 6.13 | Asset access policy | Public alongside thumbnail | Standard signed-URL pattern | +| 6.14 | Source of v1 models | **Generation only (PolyGen / Meshy)** | Uploads deferred to Phase 3 | +| 6.15 | Reactions on Model3D | Skip; users react to thumbnail Image | Saves a table + plumbing | +| 6.16 | Download tracking | ClickHouse events, no Postgres table | Denormalized into `Model3DMetric.downloadCount` | +| 6.17 | Versioning | None in v1 | Future: add `Model3DVersion` if demand emerges | +| 6.18 | File format storage | One `Model3DFile` per format, `isPrimary` for default | Unique on `(model3dId, format)` | +| 6.19 | Thumbnail source | PolyGenOutput.thumbnail, fallback to sourceImage, fallback to placeholder | Scanned through standard Image pipeline | +| 6.20 | Phase 1 effort | **L** (honest) | three.js install + viewer + handler + form + result handler + detail page + reviews UI + jobs + 15+ enum sites | +| 6.21 | Reaction-on-thumbnail UX leaks | **Accepted for pilot** | Mitigations cost ~1 week; revisit post-pilot if usage shows it matters | +| 6.22 | Review-with-images linkage | New `Post.model3dReviewId Int? @unique` | Modal optionally creates a Post for image attachments | +| 6.23 | Mod tooling for 3D | **Thumbnail-driven** | Image mod action affords "also unpublish parent Model3D"; no 3D-specific queue in v1 | +| 6.24 | Feature flag split | `model3d-feed` + `model3d-generator` (both mod-only) | Feed access ≠ generation access | +| 6.25 | Buzz pricing for PolyGen | Orchestrator `whatif` query param | Debounced call on form change; no billing infra | +| 6.26 | Generation Details params | Show params likely common across 3D gen providers | prompt, topology, polycount, symmetry, PBR, mode, seed, rigging, animation, texture prompt, source image. Hide provider-specific (e.g. `enablePromptExpansion`). | + +--- + +## 9. Revision History + +- **Rev 1** (2026-05-26): initial draft. New-entity, STL+auto-GLB, user-thumbnail, Meshy deferred. Estimated Phase 1 M–L. +- **Rev 2** (2026-05-26): critical review surfaced (a) schema is entity-enumerated not polymorphic; (b) no proven Node STL→GLB library; (c) auto-Post pattern was invented, not mirrored; (d) Phase 1 was actually L–XL. Split Phase 1 into 1a/1b/1c. Added §6.7–6.13. +- **Rev 3** (2026-05-26): decisions locked. GLB derivation dropped. ResourceReview skipped (then). Policies added as §5. +- **Rev 4** (2026-05-26): versioning removed. `Model3DVersion` and friends deleted; thumbnail/license/files moved to `Model3D`. `Post.model3dVersionId` → `Post.model3dId`. +- **Rev 5** (2026-05-27): pivoted to **generation-only**. User uploads deferred to Phase 3 (schema is upload-ready). Dropped `Model3DReaction` (users react to thumbnail Image) and `Model3DDownloadHistory` (ClickHouse handles download events). **Reviews back in scope** with `Model3DReview` + `/3d-models/[id]/reviews` route. `Model3DFile.type` enum → `format` String to accept any Meshy output format. Added `Model3D.workflowId`, `sourceImageId`, `generationParams`. New routes: `/3d-models`, `/3d-models/[id]`, `/3d-models/[id]/reviews`. Tag taxonomy genericized (no print-specific tags). PolyGen integration documented (text-to-3D + image-to-3D via Meshy/Fal). +- **Rev 6** (2026-05-27): `@civitai/client@0.2.0-beta.67` installed — exports `invokePolyGenStepTemplate`, `Meshy*PolyGenInput` variants, `Model3dBlob`. Orchestrator client dependency cleared; Phase 1's generation form is no longer blocked. Removed "Orchestrator PolyGen rollout" from the risk register. +- **Rev 7** (2026-05-27): storage no longer a constraint per product direction. Dropped 500 MB per-file CHECK from migration and app-layer enforcement plan. Removed "Storage / egress" from the risk register. File size cap decision log entry (6.3) updated to "no cap in v1". +- **Rev 8** (2026-05-27): third-review tightening pass. **Schema**: `Model3D.workflowId` → UNIQUE (prevents dup Post-from-Generation), `Model3D.scannedAt` dropped (no defined writer; only thumbnail is scanned), `Model3DFile.virusScanResult` default → `'Success'` (orchestrator-trusted in v1), added `Model3DMetric.reactionCount` denormalized from the thumbnail's `ImageMetric`. **Plan**: §2.2 corrected to use `submitWorkflow` + `PolyGenStep` (async workflow) instead of `invokePolyGenStepTemplate` (sync); §2.4 acknowledges three.js viewer is from-scratch + requires install; §2.11 spells out the "Post from Generation" flow matching `GeneratedImageActions.tsx` pattern; queue cards show thumbnails (no inline WebGL); §3 touch list adds `ReportEntity` enum, `SearchIndexEntityTypes` (PascalCase), NSFW propagation job, metrics rollup job; §6 acknowledges reaction-on-thumbnail UX leak as accepted trade-off with mitigation costs. Diagram and summary aligned. +- **Rev 9** (2026-05-27): open questions resolved with product. Phase 1 acknowledged as **L** (not M). Reaction-on-thumbnail leaks accepted for pilot. **Reviews are a modal with image attachments**: new `Post.model3dReviewId Int? @unique` lets a review own a Post that carries its attached images; this formalizes the AI-side "loose Post↔Review connection" as a stronger link for 3D. **Mod tooling**: thumbnail-driven — image-mod page gains "also action parent Model3D" affordance, no 3D-specific queue in v1. **Two Flipt flags**: `model3d-feed` (view/comment/review) + `model3d-generator` (create). **Buzz pricing**: orchestrator's `whatif` query param, no separate billing infra. **Generation Details**: surface provider-agnostic 3D-gen params (prompt, topology, polycount, symmetry, PBR, mode, seed, rigging, animation, texture prompt, source image). Schema.prisma is **source of truth** — migrations are generated from it, not the other way around; re-applying all Model3D additions to the schema now. diff --git a/docs/3d-models-summary.md b/docs/3d-models-summary.md new file mode 100644 index 0000000000..00afccab78 --- /dev/null +++ b/docs/3d-models-summary.md @@ -0,0 +1,71 @@ +# 3D Models on Civitai — Pilot Summary + +**One-liner**: Add a 3D Model content type to Civitai populated by **AI generation** (Meshy via the orchestrator's PolyGen recipe). Hackathon pilot. User uploads deferred — schema is upload-ready for later. + +--- + +## What users get in v1 + +- **Generate a 3D model** from the generation panel — text-to-3D or image-to-3D (Meshy via PolyGen). +- View the generated model in-browser with rotation/zoom. +- Click **"Post from Generation"** to publish it as a `Model3D` (creates the showcase Post with the generator's thumbnail). +- Browse a dedicated 3D Models feed and per-creator profile tab. +- Download in multiple formats (GLB, FBX, OBJ, USDZ — whatever Meshy outputs) via a dropdown selector. +- Comment, react to the thumbnail, report, tip. +- **Rate + review** with stars and recommendation. +- Community members can post "Makes/Uses" (e.g. "I used this model in my game") with photos linked back to the Model3D. + +## What's deliberately out of v1 + +- **User uploads** — schema supports them (nullable `workflowId`), code path doesn't yet. Phase 3 follow-up once we have a 3D-content moderation plan. +- Slicer integration, marketplace, paid downloads, bulk-import — all follow-ups. +- Versioning — a Model3D is atomic; iteration means a new Model3D. +- Reactions on the Model3D itself — users react to the thumbnail Image, which is already an Image row. + +## Routes + +| Route | View | +|---|---| +| `/3d-models` | Feed (Meilisearch-backed) | +| `/3d-models/[id]` | Detail: preview, info, files dropdown, generation details, comments, makes/uses | +| `/3d-models/[id]/reviews` | Reviews list + write-review CTA | +| Generation panel | New "3D Model" content type (text-to-3D + image-to-3D tabs) | + +## Phasing & effort + +| Phase | Scope | Effort | +|---|---|---| +| **1** | Schema + generation panel + save-as-Model3D + detail page + reviews (with image attachments) | **L** | +| **2** | Public feed broadening, profile tab, community "Makes/Uses" Posts | S–M | +| **3** | User uploads (deferred — needs moderation plan) | M–L | + +**Feature flags**: `model3d-feed` (browse/comment/review) and `model3d-generator` (create) — both Flipt-managed, mod-only at launch, broadened independently. + +**Unblocked** — `@civitai/client@0.2.0-beta.67` (installed) exports `submitWorkflow`, `PolyGenStep`/`PolyGenStepTemplate`, and the Meshy input types. Phase 1 can start in full; integration uses the async workflow pattern (same as Sora/Wan), not the sync template endpoint. + +## Content policy (decided) + +- **Weapons / firearms**: total ban — orchestrator prompt filter + mod review. +- **POI / real-person likenesses**: standard Civitai POI rules apply to the thumbnail Image + the prompt. +- **NSFW**: gated by orchestrator's `allowMatureContent`; only the thumbnail Image is content-scanned in v1. +- **Copyrighted IP**: report-driven into mod queue via `Model3DReport`. + +## Top risks + +1. **Meshy cost per generation**: Buzz pricing model needs sign-off from billing. +2. **3D moderation surface**: only the thumbnail is content-scanned. Mature prompts with sanitized thumbnails slip past; `allowMatureContent` is the primary mitigation. +3. **Generation reliability**: Meshy can fail/time-out. UX needs a clear retry path. +4. **Scope creep**: marketplace / user uploads are tempting — explicitly out of v1. + +_Storage / egress is intentionally not a top risk in v1 — product direction is to run rampant on storage and revisit if costs spike._ + +## Strategic value + +- **Closes the multimodal loop**: image + video + audio + **3D**, all from one generation panel. +- **No competing AI-native site does end-to-end 3D**: Thingiverse/Printables/MakerWorld are upload-only and not AI-aware. Meshy is generation-only with no community surface. +- **Low ingestion friction**: by skipping uploads in v1 we sidestep the 3D-moderation problem entirely while still validating audience interest. + +--- + +**Full plan**: `docs/3d-models-plan.md` +**Diagrams**: `docs/3d-models-diagram.md` diff --git a/docs/plans/boogu-generation-support.md b/docs/plans/boogu-generation-support.md new file mode 100644 index 0000000000..ccbcdfd510 --- /dev/null +++ b/docs/plans/boogu-generation-support.md @@ -0,0 +1,174 @@ +# Boogu — Generation Support Plan + +Status: **PREPPED, not executed.** Ecosystem record already merged (PR #2656: `ECO.Boogu=74`, `BM.Boogu=93`, family 23, Apache-2.0 license id 13). This doc outlines everything needed to wire Boogu into the generator so we can execute fast once the blockers clear. + +## What Boogu is + +`Boogu-Image-0.1` — unified multimodal image gen + edit. Built on Qwen3-VL-8B-Instruct (understanding) + FLUX.1-dev (generation), ~10B params, Apache-2.0. Three published checkpoints: + +| Variant | Task | Steps | CFG | Notes | +|---|---|---|---|---| +| **Base** | text-to-image | 25-50 (def ~35) | 2.0-5.0 (def 4) | text rendering + controllability | +| **Turbo** | text-to-image | 4 | 0.0 | Decoupled-DMD distilled, fast | +| **Edit** | text-guided image-to-image | 25-50 | 2.0-5.0 (def 5) | object/attribute/style edits | + +Default resolution 1024x1024 (2K capable). Bilingual (CN/EN) text rendering. + +--- + +## BLOCKERS (must clear before generation works end-to-end) + +1. **No `@civitai/client` types for Boogu.** Checked installed (`0.2.0-beta.71`), `latest` tag (`0.1.1-beta.0`), newest `beta` (`0.2.0-beta.72`) — zero `boogu`. ZImage types exist there; Boogu does not. **The orchestrator team must add a Boogu engine and publish typed inputs** (`BooguBaseCreateImageGenInput` / `BooguTurboCreateImageGenInput` / `BooguEditImageGenInput`, names TBD) before the handler can be strictly typed and before whatIf/generation actually succeeds. Until then we can scaffold the graph + UI and a handler with generic `ImageGenStepTemplate` + `as` casts, but it won't run. +2. **Civitai model version IDs — RESOLVED.** The 3 CivitaiOfficial draft checkpoints + their v0.1 version IDs (graph discriminator + `ecosystemSettings.defaults.model.id`): + + | Variant | Model ID | Version ID | + |---|---|---| + | Base | 2714299 | **3049541** | + | Edit | 2714541 | **3049824** | + | Turbo | 2714686 | **3050010** | + + ```ts + const booguVersionIds = { base: 3049541, edit: 3049824, turbo: 3050010 } as const; + ``` + + Base-model flip (Other -> Boogu) script: `C:\Users\Zipp4\AppData\Local\Temp\boogu-flip.mjs` (dry-run default; `--execute` to write). Validated via dry-run; run after the v5.0.1868 deploy is live in prod. +3. **Engine string + edit operation contract — orchestrator's call.** ZImage uses `engine: 'sdcpp', ecosystem: 'zImage'`. Boogu's engine (comfy? sdcpp? a new one?) and whether edit is `operation: 'editImage'` vs image-presence-inferred is whatever the orchestrator implements. Confirm with orchestrator team. + +## Gating mechanism (answer to "Flipt or ecosystem mgmt?") + +**Not Flipt.** Recent ecosystems (ZImage, Ideogram, Ernie, Krea2) have **no** Flipt flags. Visibility is gated by flags on the `BaseModelRecord` in `basemodel.constants.ts`: +- `hidden: true` — not user-facing yet (Imagen4, NanoBanana, OpenAI, SCascade all use this) +- `disabled: true` — root-level disable of all support +- `experimental: true` — shown but flagged experimental (Qwen) + +Plan: ship the graph/handler with `BM.Boogu` marked `hidden: true`, flip to visible when orchestrator + versions are ready. No Flipt work needed. + +--- + +## Architecture decision: ONE ecosystem, model.id discriminator + +You wanted a single "Boogu" ecosystem. That fits the modern pattern and we don't need ZImage's two-ecosystem split. + +- **ZImage** modeled Turbo/Base as **two ecosystems** (`ZImageTurbo`, `ZImageBase`) sharing one graph, discriminated on `ctx.ecosystem`. +- **NanoBanana / Qwen2 / Flux2** model **multiple modes in ONE ecosystem**, discriminated on `model.id`, and handle txt2img-vs-edit in the same ecosystem by appearing in both `TXT2IMG_IDS` and `EDIT_IMG_IDS` with a handler that branches on image presence. + +**Recommendation: mirror NanoBanana.** One `Boogu` ecosystem, computed discriminator on selected `model.id` → 3 subgraphs (Base / Turbo / Edit), each declaring its own `cfgScale`/`steps` `sliderNode` defaults+ranges. Edit subgraph enables the `images` node; Base/Turbo don't. Register `ECO.Boogu` in both `TXT2IMG_IDS` (Base/Turbo) and `EDIT_IMG_IDS` (Edit). Handler sets `operation: 'editImage'` when `hasImages`, else `'createImage'`. + +This keeps it to the single ecosystem you asked for while supporting all 3 variants. + +--- + +## File-by-file changes + +### 1. `src/shared/data-graph/generation/boogu-graph.ts` (NEW) + +Mirror `nano-banana-graph.ts` (3-mode, model.id discriminator) + `z-image-graph.ts` (turbo slider ranges). Structure: + +```ts +const booguVersionIds = { + base: , // BLOCKER #2 + turbo: , + edit: , +} as const; + +// Turbo subgraph: cfgScale {min:1,max:2,step:0.1,default:1} (orchestrator clamps CFG 0) +// steps {min:1,max:12,default:4}; NO sampler/scheduler; NO images node +// Base subgraph: cfgScale {min:1,max:8,step:0.5,default:4} +// steps {min:1,max:50,default:35}; NO images node +// Edit subgraph: cfgScale {min:1,max:8,step:0.5,default:5} +// steps {min:1,max:50,default:35} +// images node: when !ctx.workflow.startsWith('txt'), imagesNode({ max: 1 }) // confirm max +// +// .computed('booguMode', ctx => mode-from-model.id, ['model']) +// .groupedDiscriminator('booguMode', [{turbo},{base},{edit}]) +``` + +- Declare slider defaults **on each subgraph** — do NOT use `.effect()` to reset across variants (clobbers stored values + fires server-side; see skill gotcha). +- Aspect ratios: use `sdxlAspectRatioBuckets` (FLUX-based), default `1:1`, 1024-centric. Add `priorityOptions: ['16:9','4:3','1:1','3:4','9:16']` if >5 ratios. +- Resources/LoRA: Boogu is FLUX.1-dev-based → **could** support community LoRAs. Default to `createResourcesGraph` merge (like ZImage) but confirm orchestrator accepts them; if not, drop. +- Export `booguVersionIds` for the handler. + +### 2. `src/server/services/orchestrator/ecosystems/boogu.handler.ts` (NEW) + +Mirror `nano-banana.handler.ts` / `flux2.handler.ts`: + +```ts +const hasImages = !!data.images?.length; +const operation = hasImages ? 'editImage' : 'createImage'; +return [{ + $type: 'imageGen', + input: removeEmpty({ + engine: '', // BLOCKER #3 + // ecosystem: 'boogu', // only if comfy/sdcpp engine + model: 'base'|'turbo'|'edit'>, + operation, // if orchestrator requires it + prompt: data.prompt, + images: hasImages ? data.images?.map(x => x.url) : undefined, + aspectRatio: data.aspectRatio?.value, + cfgScale: 'cfgScale' in data ? data.cfgScale : undefined, + steps: 'steps' in data ? data.steps : undefined, + seed: data.seed, + }) as , // BLOCKER #1 — generic ImageGenStepTemplate cast until client ships +}]; +``` + +### 3. `src/shared/constants/basemodel.constants.ts` + +- **`ecosystemSupport`**: add `{ ecosystemId: ECO.Boogu, supportType: 'generation', modelTypes: checkpointAndLora }` (+ training `loraOnly` + auction if we allow LoRA training; else `checkpointOnly`). +- **`ecosystemSettings`**: `{ ecosystemId: ECO.Boogu, defaults: { model: { id: } } }`. +- **`BM.Boogu` record**: add `hidden: true` until launch-ready (BLOCKER gate). + +### 4. `src/shared/data-graph/generation/config/workflows.ts` + +- Add `ECO.Boogu` to **`TXT2IMG_IDS`** (Base/Turbo). +- Add `ECO.Boogu` to **`EDIT_IMG_IDS`** (Edit). +- Add `img2img:edit` + `txt2img` entries to `NEW_FORM_ONLY` (every new ecosystem is new-form-only): + ```ts + ['txt2img', (ecoId) => /* ... */ ecoId === ECO.Boogu], + ['img2img:edit', (ecoId) => /* ... */ ecoId === ECO.Boogu], + ``` + +### 5. `src/shared/data-graph/generation/ecosystem-graph.ts` + +- `import { booguGraph } from './boogu-graph';` +- Add to `groupedDiscriminator` (image group): `{ values: ['Boogu'] as const, graph: booguGraph },` + +### 6. `src/server/services/orchestrator/ecosystems/index.ts` + +- `import { createBooguInput } from './boogu.handler';` +- `export type BooguCtx = EcosystemGraphOutput & { ecosystem: 'Boogu' };` +- `export { createBooguInput } from './boogu.handler';` +- switch case: `case 'Boogu': return createBooguInput(normalizedData, handlerCtx);` + +### 7. `src/components/generation_v2/GenerationFormProvider.tsx` + +- Add `'Boogu'` to `TURBO_VARIANT_ECOSYSTEMS` so `cfgScale`/`steps` scope per `model.id` — Turbo (cfg 0, 4 steps) and Base (cfg 4, ~35 steps) have very different ranges within the one ecosystem and would otherwise trample each other's stored values. + +### 8. Typecheck + +`pnpm run typecheck` — iterate to clean. Expect type-cast friction from BLOCKER #1 (no client types); use generic `ImageGenStepTemplate` + `as` until the client ships. + +--- + +## UI behavior (free, no extra work) + +The image-upload control renders automatically: the Edit subgraph's `images` node (`when: !ctx.workflow.startsWith('txt')`) makes `GenerationForm.tsx`'s `Controller name="images"` render `ImageUploadMultipleInput`. Mask drawing enables when `workflow === 'img2img:edit'`. Driven purely by graph structure + workflow-array membership — no ecosystem flag needed. + +--- + +## Execution order (once blockers clear) + +1. Orchestrator ships Boogu engine + publishes `@civitai/client` with Boogu types. → `pnpm add @civitai/client@` +2. Get the 3 Civitai model version IDs from Justin. +3. Confirm engine string + edit contract + LoRA support with orchestrator. +4. Make edits 1-7 in one pass; typecheck. +5. Verify in local form (dev-server): select Boogu, switch Base/Turbo/Edit, confirm controls + whatIf. +6. Flip `BM.Boogu` off `hidden` to launch. + +## Open questions for orchestrator team + +- Engine string for Boogu? (`comfy` / `sdcpp` / new?) +- Edit: `operation: 'editImage'` field, or inferred from image presence? +- Community LoRAs on the FLUX.1-dev base — supported? +- Edit input: single source image or multiple? (assumed `max: 1`) +- Turbo CFG truly 0 — does the slider send 0 or does orchestrator hardcode it? diff --git a/package.json b/package.json index 0e08d48508..531adc2d42 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "model-share", - "version": "5.0.1861", + "version": "5.0.1885", "private": true, "packageManager": "pnpm@10.28.1", "scripts": { @@ -11,6 +11,7 @@ "predev": "pnpm build:workers", "dev": "next dev", "dev:auth": "pnpm --filter @civitai/auth-app dev", + "dev-low": "cross-env NODE_OPTIONS=\"--max_old_space_size=6144\" next dev", "dev-debug": "pnpm build:workers && cross-env NODE_OPTIONS=\"--max_old_space_size=8192 --inspect\" next dev", "dev-snap": "cross-env NODE_OPTIONS=\"--max_old_space_size=8192 --heapsnapshot-near-heap-limit=3\" next dev", "dev:daemon": "node .claude/skills/dev-server/console.mjs", @@ -88,7 +89,7 @@ "@civitai/app-sdk": "^0.6.0", "@civitai/auth": "workspace:*", "@civitai/blocks-react": "^0.4.0", - "@civitai/client": "0.2.0-beta.71", + "@civitai/client": "0.2.0-beta.73", "@civitai/cybertipline-tools": "^0.1.0", "@civitai/next-axiom": "^0.17.0", "@clavata/sdk": "^0.2.3", @@ -267,6 +268,7 @@ "stream-to-blob": "^2.0.1", "stripe": "^11.6.0", "superjson": "^2.2.6", + "three": "^0.180.0", "trie-memoize": "^1.2.0", "unfurl.js": "^6.4.0", "unified": "^11.0.5", @@ -309,6 +311,7 @@ "@types/sanitize-html": "^2.6.2", "@types/semver": "^7.7.1", "@types/sharp": "^0.31.0", + "@types/three": "^0.180.0", "@types/uuid": "^9.0.0", "@types/vimeo__player": "^2.18.3", "@types/xml2js": "^0.4.14", diff --git a/packages/civitai-axiom/src/__tests__/logToAxiom.test.ts b/packages/civitai-axiom/src/__tests__/logToAxiom.test.ts new file mode 100644 index 0000000000..daa410bb69 --- /dev/null +++ b/packages/civitai-axiom/src/__tests__/logToAxiom.test.ts @@ -0,0 +1,108 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { createAxiomLogger } from '../client'; + +// Pins the ordering contract of `logToAxiom`: the structured stderr line (which Loki ingests — it carries +// message/stack/code/path) must be emitted BEFORE the `if (!axiom) return` / `if (!datastream) return` +// guards. Previously it sat after those guards, so preview environments — where the Axiom token is a +// placeholder → `axiom` is null → the function returned early — never reached Loki (confirmed: 0 +// INTERNAL_SERVER_ERROR matches across 2.2M preview log lines). The same blind spot hits any Axiom outage. +// +// Relocated from the main app's src/server/logging/__tests__/client.test.ts when the logger moved into this +// package: the factory takes a Partial, so we inject isProd/token/datastream/logErrorsToStdout +// directly instead of mocking the app's `~/env/*` modules (which the package never reads). + +const h = vi.hoisted(() => ({ ingestEvents: vi.fn() })); +vi.mock('@axiomhq/axiom-node', () => ({ + Client: class { + ingestEvents = h.ingestEvents; + }, +})); + +const ERR = { + message: 'boom', + stack: 'Error: boom\n at x', + code: 'INTERNAL_SERVER_ERROR', + path: 'model.getById', +}; + +describe('logToAxiom stderr-for-Loki ordering', () => { + let errorSpy: ReturnType; + + beforeEach(() => { + h.ingestEvents.mockReset().mockResolvedValue(undefined); + errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + }); + + afterEach(() => { + errorSpy.mockRestore(); + }); + + it('REGRESSION GUARD: emits the structured stderr line in prod even when Axiom is null (preview/outage)', async () => { + // No token → axiom client is null (the preview/outage case that was broken). + const { logToAxiom } = createAxiomLogger({ + isProd: true, + token: undefined, + orgId: undefined, + datastream: undefined, + podName: 'pod-test', + logErrorsToStdout: true, + }); + + await logToAxiom(ERR); + + expect(errorSpy).toHaveBeenCalledTimes(1); + const payload = errorSpy.mock.calls[0][0] as string; + expect(typeof payload).toBe('string'); + const parsed = JSON.parse(payload); + expect(parsed).toMatchObject({ + message: ERR.message, + stack: ERR.stack, + code: ERR.code, + path: ERR.path, + }); + // No datastream in previews → `_axiom` is undefined and dropped by JSON.stringify. + expect('_axiom' in parsed).toBe(false); + // Axiom ingest is correctly skipped when no client is configured. + expect(h.ingestEvents).not.toHaveBeenCalled(); + }); + + it('does NOT emit the stderr line when logErrorsToStdout is false (no stderr spam)', async () => { + const { logToAxiom } = createAxiomLogger({ + isProd: true, + token: undefined, + orgId: undefined, + logErrorsToStdout: false, + }); + + await logToAxiom(ERR); + + expect(errorSpy).not.toHaveBeenCalled(); + expect(h.ingestEvents).not.toHaveBeenCalled(); + }); + + it('still ingests to Axiom (existing behavior) AND emits the stderr line when configured + flag on', async () => { + const { logToAxiom } = createAxiomLogger({ + isProd: true, + token: 'token', + orgId: 'org', + datastream: 'civitai-errors', + podName: 'pod-test', + logErrorsToStdout: true, + }); + + await logToAxiom(ERR); + + // stderr line emitted, carrying the resolved datastream in `_axiom`. + expect(errorSpy).toHaveBeenCalledTimes(1); + const parsed = JSON.parse(errorSpy.mock.calls[0][0] as string); + expect(parsed._axiom).toBe('civitai-errors'); + expect(parsed).toMatchObject({ message: ERR.message, code: ERR.code }); + + // Axiom ingest preserved. + expect(h.ingestEvents).toHaveBeenCalledTimes(1); + expect(h.ingestEvents).toHaveBeenCalledWith( + 'civitai-errors', + expect.objectContaining({ message: ERR.message, code: ERR.code, pod: 'pod-test' }) + ); + }); +}); diff --git a/packages/civitai-axiom/src/client.ts b/packages/civitai-axiom/src/client.ts index de1a19c6f6..eac9368b5e 100644 --- a/packages/civitai-axiom/src/client.ts +++ b/packages/civitai-axiom/src/client.ts @@ -48,19 +48,23 @@ export function createAxiomLogger(overrides: Partial = {}): AxiomLo async function logToAxiom(data: MixedObject, datastream?: string) { const sendData = { pod: config.podName, ...data }; if (config.isProd) { - if (!axiom) return; datastream ??= config.datastream; - if (!datastream) return; - // Write stderr BEFORE awaiting Axiom — when Axiom is degraded, - // ingestEvents rejects and the rest of this function never runs. - // Loki ingest depends on the stderr line; without this ordering, - // the Grafana alerts that consume `{ "name": "sysredis-fail-open", - // ... }` go silent during the exact multi-service incident class - // they exist to handle (sysRedis + Axiom both down). + // Write stderr BEFORE the Axiom-null/datastream guards (and before awaiting + // Axiom) — Loki ingest depends on this stderr line, so it must fire even when + // no Axiom client is configured (preview envs use a placeholder token → axiom + // is null) or when Axiom is degraded (ingestEvents rejects and the rest of this + // function never runs). Without this ordering, preview tRPC 500s never reach + // Loki, and the Grafana alerts that consume `{ "name": "sysredis-fail-open", + // ... }` go silent during the exact multi-service incident class they exist to + // handle (sysRedis + Axiom both down). `_axiom: datastream` may be undefined in + // previews (no AXIOM_DATASTREAM) — JSON.stringify drops it; the line still + // carries message/stack/code/path. if (config.logErrorsToStdout) console.error(JSON.stringify({ _axiom: datastream, ...sendData })); + if (!axiom) return; + if (!datastream) return; await axiom.ingestEvents(datastream, sendData); } else { console.log('logToAxiom', sendData); diff --git a/packages/civitai-redis/vitest.config.ts b/packages/civitai-axiom/vitest.config.mts similarity index 100% rename from packages/civitai-redis/vitest.config.ts rename to packages/civitai-axiom/vitest.config.mts diff --git a/packages/civitai-db-schema/prisma/schema.full.prisma b/packages/civitai-db-schema/prisma/schema.full.prisma index bfe78af25e..b257a0b206 100644 --- a/packages/civitai-db-schema/prisma/schema.full.prisma +++ b/packages/civitai-db-schema/prisma/schema.full.prisma @@ -548,6 +548,12 @@ model User { generationPresets GenerationPreset[] ownedWildcardSets WildcardSet[] @relation("WildcardSetOwner") + // 3D Models + model3ds Model3D[] @relation("model3dCreator") + deletedModel3Ds Model3D[] @relation("model3dDeletedBy") + model3dEngagements Model3DEngagement[] + model3dReviews Model3DReview[] + // Comics comicProjects ComicProject[] comicReferences ComicReference[] @@ -568,6 +574,7 @@ model User { publishRequestsReviewed AppBlockPublishRequest[] @relation("PublishRequestReviewer") blockScopeInvocations BlockScopeInvocation[] appUserScopeGrants AppUserScopeGrant[] + appBlockReviews AppBlockReview[] appDevForgejoIdentity AppDevForgejoIdentity? @@index([deletedAt]) @@ -1353,6 +1360,8 @@ model Report { chat ChatReport? comicProject ComicProjectReport? automated ReportAutomated? + model3d Model3DReport? + model3dReview Model3DReviewReport? } model ResourceReviewReport { @@ -1542,6 +1551,10 @@ model Post { user User @relation(fields: [userId], references: [id], onDelete: Cascade) modelVersionId Int? modelVersion ModelVersion? @relation(fields: [modelVersionId], references: [id], onDelete: SetNull) + model3dId Int? + model3d Model3D? @relation(fields: [model3dId], references: [id], onDelete: SetNull) + model3dReviewId Int? @unique + model3dReview Model3DReview? @relation(fields: [model3dReviewId], references: [id], onDelete: SetNull) createdAt DateTime @default(now()) updatedAt DateTime @updatedAt publishedAt DateTime? @@ -1568,6 +1581,7 @@ model Post { collectionItems CollectionItem[] @@index([modelVersionId]) + @@index([model3dId]) @@index([publishedAt]) } @@ -1697,6 +1711,8 @@ model Image { comicProjectHero ComicProject[] @relation("comicProjectHero") challengesCover Challenge[] @relation("ChallengeCoverImage") challengeWins ChallengeWinner[] + model3dThumbnails Model3D[] @relation("model3dThumbnail") + model3dSources Model3D[] @relation("model3dSource") @@index([featuredAt]) @@index([postId], type: Hash) @@ -1909,6 +1925,7 @@ enum TagTarget { Article Bounty Collection + Model3D } enum TagType { @@ -1953,6 +1970,7 @@ model Tag { tagsOnBounties TagsOnBounty[] CollectionItem CollectionItem[] tagsOnImage TagsOnImageDetails[] + tagsOnModel3D TagsOnModel3D[] @@unique([name]) } @@ -2303,6 +2321,7 @@ model AppBlock { publishRequests AppBlockPublishRequest[] scopeInvocations BlockScopeInvocation[] userScopeGrants AppUserScopeGrant[] + reviews AppBlockReview[] @@unique([appId, blockId], map: "app_blocks_app_block_uniq") // W1 audit C-3 fix: enforce one app per slug at the DB layer. The @@ -2316,6 +2335,40 @@ model AppBlock { @@map("app_blocks") } +/// App Blocks marketplace review (F-E "marketplace" cluster). A parallel table +/// to ResourceReview (which is hard-bound to model/version FKs) so we can carry +/// star ratings for app blocks without bending the model-review shape. +/// +/// 5-star rating (1..5, validated in the service). A blue-buzz reward fires +/// ONCE per (user, app) on the first create. Self-reviews (app owner reviewing +/// their own app) are rejected at the service layer AND excluded from the +/// aggregate. `exclude`/`tosViolation` are moderator controls that keep abusive +/// reviews out of the rating aggregate + the Bayesian marketplace sort. +/// +/// ADDITIVE + MANUALLY APPLIED (CNPG nvme0 does not auto-apply; see the +/// migration). NULL/empty until applied — read only by the dark, mod-gated +/// marketplace + the gated review procedures. +model AppBlockReview { + id Int @id @default(autoincrement()) + appBlockId String @map("app_block_id") + appBlock AppBlock @relation(fields: [appBlockId], references: [id], onDelete: Cascade) + userId Int @map("user_id") + user User @relation(fields: [userId], references: [id], onDelete: Cascade) + rating Int // 1..5 (STARS — validated in the service) + recommended Boolean @default(true) + details String? + exclude Boolean @default(false) + tosViolation Boolean @default(false) @map("tos_violation") + createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz(6) + updatedAt DateTime @updatedAt @map("updated_at") @db.Timestamptz(6) + + // One review per (user, app). + @@unique([appBlockId, userId], map: "app_block_reviews_app_user_uniq") + // Aggregate read path (AVG/COUNT WHERE NOT exclude). + @@index([appBlockId, exclude], map: "app_block_reviews_app_agg_idx") + @@map("app_block_reviews") +} + /// W1 v0 publish request — every version of every app goes through the /// moderator review queue. The dev uploads a ZIP via /apps/submit (first /// version) or /apps//submit-version (subsequent). civitai-web @@ -2476,6 +2529,11 @@ model BlockUserSubscription { // express partial unique indexes with array expressions inline). One // blanket sub per (user, app, scope); one pinned sub per // (user, app, scope, slot, target_model_ids[1]). + + // Phase 0 author analytics: installs-over-time for an owned app block + // (app_block_id equality + created_at range). See migration + // 20260621120000_bus_app_block_created_analytics_idx. + @@index([appBlockId, createdAt(sort: Desc)], map: "bus_app_block_created_idx") @@map("block_user_subscriptions") } @@ -3014,6 +3072,10 @@ model Thread { comicChapter ComicChapter? @relation(fields: [comicProjectId, comicChapterPosition], references: [projectId, position], onDelete: SetNull, onUpdate: Cascade) challengeId Int? @unique challenge Challenge? @relation(fields: [challengeId], references: [id], onDelete: SetNull) + model3dId Int? @unique + model3d Model3D? @relation(fields: [model3dId], references: [id], onDelete: SetNull) + model3dReviewId Int? @unique + model3dReview Model3DReview? @relation(fields: [model3dReviewId], references: [id], onDelete: SetNull) metadata Json @default("{}") // unused commentCount Int @default(0) @@ -3212,6 +3274,7 @@ enum CosmeticEntity { Image Article Post + Model3D } model UserCosmetic { @@ -3506,6 +3569,7 @@ enum CollectionType { Article Post Image + Model3D } enum CollectionMode { @@ -3567,6 +3631,8 @@ model CollectionItem { image Image? @relation(fields: [imageId], references: [id], onDelete: Cascade) modelId Int? model Model? @relation(fields: [modelId], references: [id], onDelete: Cascade) + model3dId Int? + model3d Model3D? @relation(fields: [model3dId], references: [id], onDelete: Cascade) addedById Int? addedBy User? @relation(fields: [addedById], references: [id], onDelete: SetNull) reviewedById Int? @@ -3578,10 +3644,11 @@ model CollectionItem { tag Tag? @relation(fields: [tagId], references: [id], onDelete: SetNull) scores CollectionItemScore[] - @@unique([collectionId, articleId, postId, imageId, modelId]) + @@unique([collectionId, articleId, postId, imageId, modelId, model3dId]) @@index([addedById], type: Hash) @@index([imageId], type: Hash) @@index([modelId], type: Hash) + @@index([model3dId], type: Hash) @@index([collectionId], type: Hash) } @@ -4237,6 +4304,7 @@ enum EntityType { UserProfile ResourceReview ChatMessage + Model3D } enum JobQueueType { @@ -6345,3 +6413,213 @@ model ScannerContentSnapshot { @@index([scanner]) } + +// ============================================================ +// 3D Models — see docs/3d-models-plan.md (rev 9) +// v1 source = orchestrator PolyGen generation (Meshy via Fal). +// Schema is upload-ready so a future user-upload flow can write +// rows with workflowId = NULL. +// No versioning. No reactions on Model3D (react on thumbnail Image +// instead). No DownloadHistory in Postgres (ClickHouse events). +// Reviews own an optional Post for image attachments. +// ============================================================ + +enum Model3DStatus { + Draft + Published + Unpublished + Deleted +} + +enum Model3DEngagementType { + Favorite + Hide + Notify +} + +model Model3DLicense { + id Int @id @default(autoincrement()) + name String @unique + description String + allowCommercialUse Boolean @default(false) + allowPrintFarm Boolean @default(false) + allowDerivatives Boolean @default(true) + allowRedistribution Boolean @default(false) + requireAttribution Boolean @default(true) + isCustom Boolean @default(false) + createdAt DateTime @default(now()) + + models Model3D[] +} + +model Model3D { + id Int @id @default(autoincrement()) + name String @db.Citext + description String? + userId Int + user User @relation("model3dCreator", fields: [userId], references: [id]) + thumbnailImageId Int? @unique + thumbnailImage Image? @relation("model3dThumbnail", fields: [thumbnailImageId], references: [id], onDelete: SetNull) + licenseId Int + license Model3DLicense @relation(fields: [licenseId], references: [id]) + licenseDetails String? + + // Generation provenance — NULL for future user-uploaded rows. + workflowId String? @unique + sourceImageId Int? + sourceImage Image? @relation("model3dSource", fields: [sourceImageId], references: [id], onDelete: SetNull) + generationParams Json? + + status Model3DStatus @default(Draft) + nsfw Boolean @default(false) + tosViolation Boolean @default(false) + poi Boolean @default(false) + minor Boolean @default(false) + unlisted Boolean @default(false) + lockedProperties String[] @default([]) + availability Availability @default(Public) + nsfwLevel Int @default(0) + meta Json @default("{}") + // Per-Model3D gallery moderation (creator/mod hide image/user/tag). No + // version dimension — `images` is a flat list of hidden image ids. + gallerySettings Json @default("{\"users\":[],\"tags\":[],\"images\":[]}") + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + publishedAt DateTime? + deletedAt DateTime? + deletedBy Int? + deletedByUser User? @relation("model3dDeletedBy", fields: [deletedBy], references: [id], onDelete: SetNull) + + files Model3DFile[] + posts Post[] + tags TagsOnModel3D[] + engagements Model3DEngagement[] + reports Model3DReport[] + reviews Model3DReview[] + threads Thread[] + collectionItems CollectionItem[] + metric Model3DMetric? + + @@index([userId, status, publishedAt(sort: Desc)]) + @@index([status, publishedAt(sort: Desc)]) + @@index([status, nsfwLevel, publishedAt(sort: Desc)]) + @@index([name]) + @@index([licenseId], type: Hash) + @@index([sourceImageId], type: Hash) +} + +model Model3DFile { + id Int @id @default(autoincrement()) + model3dId Int + model3d Model3D @relation(fields: [model3dId], references: [id], onDelete: Cascade) + name String + url String + sizeKB Float + format String + // Variant discriminator. Lets a single Model3D carry multiple glb/fbx + // exports for the same generation — base, rigged, animated, walking + // (with armature sibling), running (with armature sibling). Defaults to + // "primary" so existing rows + non-PolyGen ingest paths are unaffected. + variant String @default("primary") + isPrimary Boolean @default(false) + metadata Json? + virusScanResult ScanResultCode @default(Success) + virusScanMessage String? + rawScanResult Json? + scannedAt DateTime? + scanRequestedAt DateTime? + exists Boolean? + createdAt DateTime @default(now()) + + @@unique([model3dId, format, variant]) + @@index([model3dId], type: Hash) +} + +model TagsOnModel3D { + model3dId Int + model3d Model3D @relation(fields: [model3dId], references: [id], onDelete: Cascade) + tagId Int + tag Tag @relation(fields: [tagId], references: [id], onDelete: Cascade) + createdAt DateTime @default(now()) + + @@id([model3dId, tagId]) + @@index([model3dId], type: Hash) + @@index([tagId], type: Hash) +} + +model Model3DEngagement { + userId Int + user User @relation(fields: [userId], references: [id], onDelete: Cascade) + model3dId Int + model3d Model3D @relation(fields: [model3dId], references: [id], onDelete: Cascade) + type Model3DEngagementType + createdAt DateTime @default(now()) + + @@id([userId, model3dId]) + @@index([model3dId], type: Hash) +} + +model Model3DReport { + model3dId Int + model3d Model3D @relation(fields: [model3dId], references: [id], onDelete: Cascade) + reportId Int @unique + report Report @relation(fields: [reportId], references: [id], onDelete: Cascade) + + @@id([reportId, model3dId]) + @@index([model3dId], type: Hash) +} + +model Model3DReview { + id Int @id @default(autoincrement()) + model3dId Int + model3d Model3D @relation(fields: [model3dId], references: [id], onDelete: Cascade) + userId Int + user User @relation(fields: [userId], references: [id], onDelete: Cascade) + recommended Boolean @default(true) + details String? + nsfw Boolean @default(false) + tosViolation Boolean @default(false) + exclude Boolean @default(false) + metadata Json? + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + thread Thread? + post Post? + reports Model3DReviewReport[] + + @@unique([model3dId, userId]) + @@index([model3dId], type: Hash) + @@index([userId], type: Hash) +} + +model Model3DReviewReport { + model3dReviewId Int + model3dReview Model3DReview @relation(fields: [model3dReviewId], references: [id], onDelete: Cascade) + reportId Int @unique + report Report @relation(fields: [reportId], references: [id], onDelete: Cascade) + + @@id([reportId, model3dReviewId]) + @@index([model3dReviewId], type: Hash) +} + +model Model3DMetric { + model3dId Int @id + model3d Model3D @relation(fields: [model3dId], references: [id], onDelete: Cascade) + downloadCount Int @default(0) + commentCount Int @default(0) + collectedCount Int @default(0) + imageCount Int @default(0) + tippedCount Int @default(0) + tippedAmountCount Int @default(0) + ratingCount Int @default(0) + recommendedCount Int @default(0) + reactionCount Int @default(0) + earnedAmount Int @default(0) + updatedAt DateTime @default(now()) + nsfwLevel Int @default(0) + userId Int @default(0) + status Model3DStatus @default(Draft) + availability Availability @default(Public) + poi Boolean @default(false) + minor Boolean @default(false) +} diff --git a/packages/civitai-db-schema/src/enums.ts b/packages/civitai-db-schema/src/enums.ts index 71302e99c4..0653cede98 100644 --- a/packages/civitai-db-schema/src/enums.ts +++ b/packages/civitai-db-schema/src/enums.ts @@ -441,6 +441,7 @@ export const TagTarget = { Article: 'Article', Bounty: 'Bounty', Collection: 'Collection', + Model3D: 'Model3D', } as const; export type TagTarget = (typeof TagTarget)[keyof typeof TagTarget]; @@ -537,6 +538,7 @@ export const CosmeticEntity = { Image: 'Image', Article: 'Article', Post: 'Post', + Model3D: 'Model3D', } as const; export type CosmeticEntity = (typeof CosmeticEntity)[keyof typeof CosmeticEntity]; @@ -622,6 +624,7 @@ export const CollectionType = { Article: 'Article', Post: 'Post', Image: 'Image', + Model3D: 'Model3D', } as const; export type CollectionType = (typeof CollectionType)[keyof typeof CollectionType]; @@ -785,6 +788,7 @@ export const EntityType = { UserProfile: 'UserProfile', ResourceReview: 'ResourceReview', ChatMessage: 'ChatMessage', + Model3D: 'Model3D', } as const; export type EntityType = (typeof EntityType)[keyof typeof EntityType]; @@ -1069,3 +1073,20 @@ export const ReviewVerdict = { } as const; export type ReviewVerdict = (typeof ReviewVerdict)[keyof typeof ReviewVerdict]; + +export const Model3DStatus = { + Draft: 'Draft', + Published: 'Published', + Unpublished: 'Unpublished', + Deleted: 'Deleted', +} as const; + +export type Model3DStatus = (typeof Model3DStatus)[keyof typeof Model3DStatus]; + +export const Model3DEngagementType = { + Favorite: 'Favorite', + Hide: 'Hide', + Notify: 'Notify', +} as const; + +export type Model3DEngagementType = (typeof Model3DEngagementType)[keyof typeof Model3DEngagementType]; diff --git a/packages/civitai-db-schema/src/kysely/enums.ts b/packages/civitai-db-schema/src/kysely/enums.ts index 8880fb8b47..b232d4ff0b 100644 --- a/packages/civitai-db-schema/src/kysely/enums.ts +++ b/packages/civitai-db-schema/src/kysely/enums.ts @@ -360,6 +360,7 @@ export const TagTarget = { Article: 'Article', Bounty: 'Bounty', Collection: 'Collection', + Model3D: 'Model3D', } as const; export type TagTarget = (typeof TagTarget)[keyof typeof TagTarget]; export const TagType = { @@ -436,6 +437,7 @@ export const CosmeticEntity = { Image: 'Image', Article: 'Article', Post: 'Post', + Model3D: 'Model3D', } as const; export type CosmeticEntity = (typeof CosmeticEntity)[keyof typeof CosmeticEntity]; export const BuzzAccountType = { @@ -509,6 +511,7 @@ export const CollectionType = { Article: 'Article', Post: 'Post', Image: 'Image', + Model3D: 'Model3D', } as const; export type CollectionType = (typeof CollectionType)[keyof typeof CollectionType]; export const CollectionMode = { @@ -641,6 +644,7 @@ export const EntityType = { UserProfile: 'UserProfile', ResourceReview: 'ResourceReview', ChatMessage: 'ChatMessage', + Model3D: 'Model3D', } as const; export type EntityType = (typeof EntityType)[keyof typeof EntityType]; export const JobQueueType = { @@ -868,3 +872,17 @@ export const ReviewVerdict = { Unsure: 'Unsure', } as const; export type ReviewVerdict = (typeof ReviewVerdict)[keyof typeof ReviewVerdict]; +export const Model3DStatus = { + Draft: 'Draft', + Published: 'Published', + Unpublished: 'Unpublished', + Deleted: 'Deleted', +} as const; +export type Model3DStatus = (typeof Model3DStatus)[keyof typeof Model3DStatus]; +export const Model3DEngagementType = { + Favorite: 'Favorite', + Hide: 'Hide', + Notify: 'Notify', +} as const; +export type Model3DEngagementType = + (typeof Model3DEngagementType)[keyof typeof Model3DEngagementType]; diff --git a/packages/civitai-db-schema/src/kysely/types.ts b/packages/civitai-db-schema/src/kysely/types.ts index 0327a3d8c9..92053f2eec 100644 --- a/packages/civitai-db-schema/src/kysely/types.ts +++ b/packages/civitai-db-schema/src/kysely/types.ts @@ -117,6 +117,8 @@ import type { WildcardSetAuditStatus, WildcardSetCategoryAuditStatus, ReviewVerdict, + Model3DStatus, + Model3DEngagementType, } from './enums'; export type Account = { @@ -292,6 +294,18 @@ export type AppBlockPublishRequest = { created_at: Generated; updated_at: Generated; }; +export type AppBlockReview = { + id: Generated; + app_block_id: string; + user_id: number; + rating: number; + recommended: Generated; + details: string | null; + exclude: Generated; + tos_violation: Generated; + created_at: Generated; + updated_at: Timestamp; +}; export type AppDevForgejoIdentity = { user_id: number; forgejo_username: string; @@ -1448,6 +1462,7 @@ export type CollectionItem = { postId: number | null; imageId: number | null; modelId: number | null; + model3dId: number | null; addedById: number | null; reviewedById: number | null; reviewedAt: Timestamp | null; @@ -2220,6 +2235,111 @@ export type Model = { allowDerivatives: Generated; allowDifferentLicense: Generated; }; +export type Model3D = { + id: Generated; + name: string; + description: string | null; + userId: number; + thumbnailImageId: number | null; + licenseId: number; + licenseDetails: string | null; + workflowId: string | null; + sourceImageId: number | null; + generationParams: unknown | null; + status: Generated; + nsfw: Generated; + tosViolation: Generated; + poi: Generated; + minor: Generated; + unlisted: Generated; + lockedProperties: Generated; + availability: Generated; + nsfwLevel: Generated; + meta: Generated; + gallerySettings: Generated; + createdAt: Generated; + updatedAt: Timestamp; + publishedAt: Timestamp | null; + deletedAt: Timestamp | null; + deletedBy: number | null; +}; +export type Model3DEngagement = { + userId: number; + model3dId: number; + type: Model3DEngagementType; + createdAt: Generated; +}; +export type Model3DFile = { + id: Generated; + model3dId: number; + name: string; + url: string; + sizeKB: number; + format: string; + variant: Generated; + isPrimary: Generated; + metadata: unknown | null; + virusScanResult: Generated; + virusScanMessage: string | null; + rawScanResult: unknown | null; + scannedAt: Timestamp | null; + scanRequestedAt: Timestamp | null; + exists: boolean | null; + createdAt: Generated; +}; +export type Model3DLicense = { + id: Generated; + name: string; + description: string; + allowCommercialUse: Generated; + allowPrintFarm: Generated; + allowDerivatives: Generated; + allowRedistribution: Generated; + requireAttribution: Generated; + isCustom: Generated; + createdAt: Generated; +}; +export type Model3DMetric = { + model3dId: number; + downloadCount: Generated; + commentCount: Generated; + collectedCount: Generated; + imageCount: Generated; + tippedCount: Generated; + tippedAmountCount: Generated; + ratingCount: Generated; + recommendedCount: Generated; + reactionCount: Generated; + earnedAmount: Generated; + updatedAt: Generated; + nsfwLevel: Generated; + userId: Generated; + status: Generated; + availability: Generated; + poi: Generated; + minor: Generated; +}; +export type Model3DReport = { + model3dId: number; + reportId: number; +}; +export type Model3DReview = { + id: Generated; + model3dId: number; + userId: number; + recommended: Generated; + details: string | null; + nsfw: Generated; + tosViolation: Generated; + exclude: Generated; + metadata: unknown | null; + createdAt: Generated; + updatedAt: Timestamp; +}; +export type Model3DReviewReport = { + model3dReviewId: number; + reportId: number; +}; export type ModelAssociations = { id: Generated; fromModelId: number; @@ -2545,6 +2665,8 @@ export type Post = { detail: string | null; userId: number; modelVersionId: number | null; + model3dId: number | null; + model3dReviewId: number | null; createdAt: Generated; updatedAt: Timestamp; publishedAt: Timestamp | null; @@ -3070,6 +3192,11 @@ export type TagsOnImageVote = { createdAt: Generated; applied: Generated; }; +export type TagsOnModel3D = { + model3dId: number; + tagId: number; + createdAt: Generated; +}; export type TagsOnModels = { modelId: number; tagId: number; @@ -3161,6 +3288,8 @@ export type Thread = { comicProjectId: number | null; comicChapterPosition: number | null; challengeId: number | null; + model3dId: number | null; + model3dReviewId: number | null; metadata: Generated; commentCount: Generated; }; @@ -3520,6 +3649,7 @@ export type DB = { AnswerVote: AnswerVote; ApiKey: ApiKey; app_block_publish_requests: AppBlockPublishRequest; + app_block_reviews: AppBlockReview; app_blocks: AppBlock; app_dev_forgejo_identity: AppDevForgejoIdentity; app_user_scope_grants: AppUserScopeGrant; @@ -3662,6 +3792,14 @@ export type DB = { Link: Link; ModActivity: ModActivity; Model: Model; + Model3D: Model3D; + Model3DEngagement: Model3DEngagement; + Model3DFile: Model3DFile; + Model3DLicense: Model3DLicense; + Model3DMetric: Model3DMetric; + Model3DReport: Model3DReport; + Model3DReview: Model3DReview; + Model3DReviewReport: Model3DReviewReport; ModelAssociations: ModelAssociations; ModelBaseModelMetric: ModelBaseModelMetric; ModelEngagement: ModelEngagement; @@ -3740,6 +3878,7 @@ export type DB = { TagsOnImageDetails: TagsOnImageDetails; TagsOnImageNew: TagsOnImageNew; TagsOnImageVote: TagsOnImageVote; + TagsOnModel3D: TagsOnModel3D; TagsOnModels: TagsOnModels; TagsOnModelsVote: TagsOnModelsVote; TagsOnPost: TagsOnPost; diff --git a/packages/civitai-db-schema/src/models.ts b/packages/civitai-db-schema/src/models.ts index fb1b39e0f4..6be35e57e2 100644 --- a/packages/civitai-db-schema/src/models.ts +++ b/packages/civitai-db-schema/src/models.ts @@ -90,7 +90,7 @@ export type ImageEngagementType = "Favorite" | "Hide"; export type ImageOnModelType = "Example" | "Training"; -export type TagTarget = "Model" | "Question" | "Image" | "Post" | "Tag" | "Article" | "Bounty" | "Collection"; +export type TagTarget = "Model" | "Question" | "Image" | "Post" | "Tag" | "Article" | "Bounty" | "Collection" | "Model3D"; export type TagType = "UserGenerated" | "Label" | "Moderation" | "System"; @@ -110,7 +110,7 @@ export type CosmeticType = "Badge" | "NamePlate" | "ContentDecoration" | "Profil export type CosmeticSource = "Trophy" | "Purchase" | "Event" | "Membership" | "Claim"; -export type CosmeticEntity = "Model" | "Image" | "Article" | "Post"; +export type CosmeticEntity = "Model" | "Image" | "Article" | "Post" | "Model3D"; export type BuzzAccountType = "user" | "generation" | "club" | "green" | "fakered"; @@ -126,7 +126,7 @@ export type CollectionWriteConfiguration = "Private" | "Public" | "Review"; export type CollectionReadConfiguration = "Private" | "Public" | "Unlisted"; -export type CollectionType = "Model" | "Article" | "Post" | "Image"; +export type CollectionType = "Model" | "Article" | "Post" | "Image" | "Model3D"; export type CollectionMode = "Contest" | "Bookmark"; @@ -160,7 +160,7 @@ export type ChatMessageType = "Markdown" | "Image" | "Video" | "Audio" | "Embed" export type PurchasableRewardUsage = "SingleUse" | "MultiUse"; -export type EntityType = "Image" | "Post" | "Article" | "Bounty" | "BountyEntry" | "ModelVersion" | "Model" | "Collection" | "Comment" | "CommentV2" | "User" | "UserProfile" | "ResourceReview" | "ChatMessage"; +export type EntityType = "Image" | "Post" | "Article" | "Bounty" | "BountyEntry" | "ModelVersion" | "Model" | "Collection" | "Comment" | "CommentV2" | "User" | "UserProfile" | "ResourceReview" | "ChatMessage" | "Model3D"; export type JobQueueType = "CleanUp" | "UpdateMetrics" | "UpdateNsfwLevel" | "UpdateSearchIndex" | "CleanIfEmpty" | "ModerationRequest" | "BlockedImageDelete" | "ImageScan"; @@ -224,6 +224,10 @@ export type WildcardSetCategoryAuditStatus = "Pending" | "Clean" | "Dirty"; export type ReviewVerdict = "TruePositive" | "FalsePositive" | "TrueNegative" | "FalseNegative" | "Unsure"; +export type Model3DStatus = "Draft" | "Published" | "Unpublished" | "Deleted"; + +export type Model3DEngagementType = "Favorite" | "Hide" | "Notify"; + export interface Account { id: number; userId: number; @@ -597,6 +601,10 @@ export interface User { voidedStrikes?: UserStrike[]; generationPresets?: GenerationPreset[]; ownedWildcardSets?: WildcardSet[]; + model3ds?: Model3D[]; + deletedModel3Ds?: Model3D[]; + model3dEngagements?: Model3DEngagement[]; + model3dReviews?: Model3DReview[]; comicProjects?: ComicProject[]; comicReferences?: ComicReference[]; comicProjectEngagements?: ComicProjectEngagement[]; @@ -614,6 +622,7 @@ export interface User { publishRequestsReviewed?: AppBlockPublishRequest[]; blockScopeInvocations?: BlockScopeInvocation[]; appUserScopeGrants?: AppUserScopeGrant[]; + appBlockReviews?: AppBlockReview[]; appDevForgejoIdentity?: AppDevForgejoIdentity | null; } @@ -1116,6 +1125,8 @@ export interface Report { chat?: ChatReport | null; comicProject?: ComicProjectReport | null; automated?: ReportAutomated | null; + model3d?: Model3DReport | null; + model3dReview?: Model3DReviewReport | null; } export interface ResourceReviewReport { @@ -1252,6 +1263,10 @@ export interface Post { user?: User; modelVersionId: number | null; modelVersion?: ModelVersion | null; + model3dId: number | null; + model3d?: Model3D | null; + model3dReviewId: number | null; + model3dReview?: Model3DReview | null; createdAt: Date; updatedAt: Date; publishedAt: Date | null; @@ -1368,6 +1383,8 @@ export interface Image { comicProjectHero?: ComicProject[]; challengesCover?: Challenge[]; challengeWins?: ChallengeWinner[]; + model3dThumbnails?: Model3D[]; + model3dSources?: Model3D[]; } export interface ImageTagForReview { @@ -1539,6 +1556,7 @@ export interface Tag { tagsOnBounties?: TagsOnBounty[]; CollectionItem?: CollectionItem[]; tagsOnImage?: TagsOnImageDetails[]; + tagsOnModel3D?: TagsOnModel3D[]; } export interface TagsOnTags { @@ -1775,6 +1793,22 @@ export interface AppBlock { publishRequests?: AppBlockPublishRequest[]; scopeInvocations?: BlockScopeInvocation[]; userScopeGrants?: AppUserScopeGrant[]; + reviews?: AppBlockReview[]; +} + +export interface AppBlockReview { + id: number; + appBlockId: string; + appBlock?: AppBlock; + userId: number; + user?: User; + rating: number; + recommended: boolean; + details: string | null; + exclude: boolean; + tosViolation: boolean; + createdAt: Date; + updatedAt: Date; } export interface AppBlockPublishRequest { @@ -2182,6 +2216,10 @@ export interface Thread { comicChapter?: ComicChapter | null; challengeId: number | null; challenge?: Challenge | null; + model3dId: number | null; + model3d?: Model3D | null; + model3dReviewId: number | null; + model3dReview?: Model3DReview | null; metadata: JsonValue; commentCount: number; comments?: CommentV2[]; @@ -2559,6 +2597,8 @@ export interface CollectionItem { image?: Image | null; modelId: number | null; model?: Model | null; + model3dId: number | null; + model3d?: Model3D | null; addedById: number | null; addedBy?: User | null; reviewedById: number | null; @@ -4609,4 +4649,153 @@ export interface ScannerContentSnapshot { createdAt: Date; } +export interface Model3DLicense { + id: number; + name: string; + description: string; + allowCommercialUse: boolean; + allowPrintFarm: boolean; + allowDerivatives: boolean; + allowRedistribution: boolean; + requireAttribution: boolean; + isCustom: boolean; + createdAt: Date; + models?: Model3D[]; +} + +export interface Model3D { + id: number; + name: string; + description: string | null; + userId: number; + user?: User; + thumbnailImageId: number | null; + thumbnailImage?: Image | null; + licenseId: number; + license?: Model3DLicense; + licenseDetails: string | null; + workflowId: string | null; + sourceImageId: number | null; + sourceImage?: Image | null; + generationParams: JsonValue | null; + status: Model3DStatus; + nsfw: boolean; + tosViolation: boolean; + poi: boolean; + minor: boolean; + unlisted: boolean; + lockedProperties: string[]; + availability: Availability; + nsfwLevel: number; + meta: JsonValue; + gallerySettings: JsonValue; + createdAt: Date; + updatedAt: Date; + publishedAt: Date | null; + deletedAt: Date | null; + deletedBy: number | null; + deletedByUser?: User | null; + files?: Model3DFile[]; + posts?: Post[]; + tags?: TagsOnModel3D[]; + engagements?: Model3DEngagement[]; + reports?: Model3DReport[]; + reviews?: Model3DReview[]; + threads?: Thread[]; + collectionItems?: CollectionItem[]; + metric?: Model3DMetric | null; +} + +export interface Model3DFile { + id: number; + model3dId: number; + model3d?: Model3D; + name: string; + url: string; + sizeKB: number; + format: string; + variant: string; + isPrimary: boolean; + metadata: JsonValue | null; + virusScanResult: ScanResultCode; + virusScanMessage: string | null; + rawScanResult: JsonValue | null; + scannedAt: Date | null; + scanRequestedAt: Date | null; + exists: boolean | null; + createdAt: Date; +} + +export interface TagsOnModel3D { + model3dId: number; + model3d?: Model3D; + tagId: number; + tag?: Tag; + createdAt: Date; +} + +export interface Model3DEngagement { + userId: number; + user?: User; + model3dId: number; + model3d?: Model3D; + type: Model3DEngagementType; + createdAt: Date; +} + +export interface Model3DReport { + model3dId: number; + model3d?: Model3D; + reportId: number; + report?: Report; +} + +export interface Model3DReview { + id: number; + model3dId: number; + model3d?: Model3D; + userId: number; + user?: User; + recommended: boolean; + details: string | null; + nsfw: boolean; + tosViolation: boolean; + exclude: boolean; + metadata: JsonValue | null; + createdAt: Date; + updatedAt: Date; + thread?: Thread | null; + post?: Post | null; + reports?: Model3DReviewReport[]; +} + +export interface Model3DReviewReport { + model3dReviewId: number; + model3dReview?: Model3DReview; + reportId: number; + report?: Report; +} + +export interface Model3DMetric { + model3dId: number; + model3d?: Model3D; + downloadCount: number; + commentCount: number; + collectedCount: number; + imageCount: number; + tippedCount: number; + tippedAmountCount: number; + ratingCount: number; + recommendedCount: number; + reactionCount: number; + earnedAmount: number; + updatedAt: Date; + nsfwLevel: number; + userId: number; + status: Model3DStatus; + availability: Availability; + poi: boolean; + minor: boolean; +} + type JsonValue = string | number | boolean | { [key in string]?: JsonValue } | Array | null; diff --git a/packages/civitai-redis/src/__tests__/cluster-deadline-hits.test.ts b/packages/civitai-redis/src/__tests__/cluster-deadline-hits.test.ts new file mode 100644 index 0000000000..e1c3e03994 --- /dev/null +++ b/packages/civitai-redis/src/__tests__/cluster-deadline-hits.test.ts @@ -0,0 +1,61 @@ +import { describe, it, expect, beforeEach } from 'vitest'; +import { + countClusterDeadlineHits, + recordClusterDeadlineHit, + resetClusterDeadlineHits, +} from '../cluster-deadline-hits'; + +// cluster-deadline-hits is the sliding-window recorder of CLUSTER command-deadline TIMEOUTS — +// the sawtooth-immune self-heal trigger signal. A healthy client records ZERO hits; a half-open +// client records one per deadline-rejected command. The watchdog samples a windowed count. +// Module-scoped state → reset before each test. + +describe('cluster-deadline-hits', () => { + beforeEach(() => resetClusterDeadlineHits()); + + it('counts hits recorded inside the window', () => { + recordClusterDeadlineHit(1000); + recordClusterDeadlineHit(1500); + recordClusterDeadlineHit(2000); + // window covering [1000..2000] from now=2000, windowMs=2000 → cutoff=0, all 3 counted + expect(countClusterDeadlineHits(2000, 2000)).toBe(3); + }); + + it('excludes hits older than the window (strictly greater than cutoff)', () => { + recordClusterDeadlineHit(0); + recordClusterDeadlineHit(5000); + recordClusterDeadlineHit(9000); + // now=10000, windowMs=2000 → cutoff=8000 → only the 9000 hit counts + expect(countClusterDeadlineHits(2000, 10000)).toBe(1); + }); + + it('returns 0 for a healthy client (no hits recorded)', () => { + expect(countClusterDeadlineHits(20000, 50000)).toBe(0); + }); + + it('reset clears the window', () => { + recordClusterDeadlineHit(1000); + recordClusterDeadlineHit(1100); + expect(countClusterDeadlineHits(10000, 2000)).toBe(2); + resetClusterDeadlineHits(); + expect(countClusterDeadlineHits(10000, 2000)).toBe(0); + }); + + it('bounds memory: keeps counting recent hits even past the ring capacity', () => { + // Record far more than RING_CAPACITY (512) hits, all within the window. The ring overwrites + // the oldest, but the count is capped at capacity — which is far above any trigger threshold, + // so the wedge is still detected. This proves a sustained wedge can't grow memory unbounded. + const now = 1_000_000; + for (let i = 0; i < 2000; i++) recordClusterDeadlineHit(now); + const count = countClusterDeadlineHits(20000, now + 1); + expect(count).toBeGreaterThanOrEqual(512); // saturated at capacity + expect(count).toBeLessThanOrEqual(512); // never exceeds capacity (bounded) + }); + + it('a sustained wedge keeps the windowed count high even as old hits age out', () => { + // Hits arriving steadily: at t=20000 with windowMs=10000, only hits in (10000,20000] count. + for (let t = 0; t <= 20000; t += 1000) recordClusterDeadlineHit(t); + // hits at 11000..20000 = 10 of them + expect(countClusterDeadlineHits(10000, 20000)).toBe(10); + }); +}); diff --git a/packages/civitai-redis/src/__tests__/cluster-inflight.test.ts b/packages/civitai-redis/src/__tests__/cluster-inflight.test.ts new file mode 100644 index 0000000000..97f1ec9c12 --- /dev/null +++ b/packages/civitai-redis/src/__tests__/cluster-inflight.test.ts @@ -0,0 +1,128 @@ +import { describe, it, expect, beforeEach } from 'vitest'; +import { + decClusterInflight, + getClusterInflight, + incClusterInflight, + resetClusterInflight, +} from '../cluster-inflight'; +import { ClusterSelfHealWatchdog } from '../cluster-selfheal'; +import type { ClusterSelfHealConfig, ClusterSelfHealDeps } from '../cluster-selfheal'; + +// FIX #2 coverage: the cluster inflight counter that the self-heal watchdog samples must +// NEVER go negative — specifically after a heal. forceClusterReconnect calls the cluster +// client's destroy() (flushAll(DisconnectsClientError)), which IMMEDIATELY rejects every +// in-flight command; each rejected command then runs done() → decClusterInflight(). Before +// FIX #2 those N decrements (a per-closure guard, no global floor) drove the counter to ≈ −N +// permanently, so the watchdog needed inflight > threshold + N to ever re-trigger → a SECOND +// wedge went unhealed. These tests exercise the REAL counter (the same module client.ts uses, +// not a stubbed constant) through the reset→decrement interaction and assert the floor holds +// and the watchdog can re-arm afterwards. + +describe('cluster-inflight counter (FIX #2 floor)', () => { + beforeEach(() => resetClusterInflight()); + + it('increments and decrements in lockstep on the happy path', () => { + expect(getClusterInflight()).toBe(0); + incClusterInflight(); + incClusterInflight(); + expect(getClusterInflight()).toBe(2); + expect(decClusterInflight()).toBe(1); + expect(decClusterInflight()).toBe(0); + expect(getClusterInflight()).toBe(0); + }); + + it('FLOORS at 0 — a decrement when already at 0 never goes negative', () => { + expect(getClusterInflight()).toBe(0); + expect(decClusterInflight()).toBe(0); + expect(decClusterInflight()).toBe(0); + expect(getClusterInflight()).toBe(0); + }); + + it('post-heal: reset()→0 then N late rejected decrements settle at 0, NOT -N', () => { + // N commands in flight when the wedge fires. + const N = 7; + for (let i = 0; i < N; i++) incClusterInflight(); + expect(getClusterInflight()).toBe(N); + + // forceClusterReconnect resets up front (the watchdog's sampled value snaps clean)... + resetClusterInflight(); + expect(getClusterInflight()).toBe(0); + + // ...then each of the N destroy()-rejected commands runs its done() → floored decrement. + // Pre-FIX-#2 this would have left the counter at -N; the floor keeps it at 0. + for (let i = 0; i < N; i++) decClusterInflight(); + expect(getClusterInflight()).toBe(0); + }); +}); + +// Integration: drive the REAL watchdog with the REAL counter as getInflight (not a constant), +// simulate a wedge → heal → post-heal rejected decrements, and prove a SECOND wedge still +// triggers a reconnect (the counter re-armed because it floored at 0 instead of going negative). +describe('ClusterSelfHealWatchdog re-arms after a heal (FIX #2, real counter)', () => { + const CFG: ClusterSelfHealConfig = { + enabled: true, + inflightThreshold: 50, + sustainedMs: 20000, + cooldownMs: 60000, + // Deadline trigger disabled (<= 0) so this suite exercises ONLY the inflight path; no + // jitter so the reconnect fires synchronously and the assertions stay deterministic. + deadlineHitThreshold: 0, + deadlineHitWindowMs: 20000, + reconnectJitterMs: 0, + }; + + beforeEach(() => resetClusterInflight()); + + it('a second wedge after a heal still triggers a reconnect (counter floored, not negative)', async () => { + let nowMs = 0; + let reconnectCount = 0; + // The reconnect models forceClusterReconnect's counter handling against the REAL counter: + // reset up front, then the N previously-in-flight commands reject and floored-decrement. + const nInFlightAtHeal = () => getClusterInflight(); + const reconnect = (): Promise => { + reconnectCount++; + const n = nInFlightAtHeal(); + resetClusterInflight(); // up-front reset + for (let i = 0; i < n; i++) decClusterInflight(); // late rejected done()s, floored + return Promise.resolve(); + }; + const deps: ClusterSelfHealDeps = { + getInflight: () => getClusterInflight(), + reconnect, + now: () => nowMs, + log: () => {}, + onReconnect: () => {}, + }; + const watchdog = new ClusterSelfHealWatchdog(CFG, deps); + + // ── First wedge: 500 commands pinned in flight ── + for (let i = 0; i < 500; i++) incClusterInflight(); + // Drive ticks across the sustained window. + for (let t = 0; t < CFG.sustainedMs; t += 1000) { + watchdog.tick(); + nowMs += 1000; + } + expect(watchdog.tick()).toBe(true); // first heal fires + await new Promise((r) => setTimeout(r, 0)); // let the reconnect settle + expect(reconnectCount).toBe(1); + // After the heal the counter floored at 0 (NOT -500). + expect(getClusterInflight()).toBe(0); + + // Advance past the cooldown so the watchdog can re-arm. + nowMs += CFG.cooldownMs + 1000; + + // ── Second wedge: another genuine pin. With a negative counter this could never reach + // threshold again; floored at 0 it can. ── + for (let i = 0; i < 500; i++) incClusterInflight(); + expect(getClusterInflight()).toBe(500); + let secondFired = false; + for (let t = 0; t < CFG.sustainedMs + 2000; t += 1000) { + if (watchdog.tick()) secondFired = true; + nowMs += 1000; + } + expect(secondFired).toBe(true); + expect(reconnectCount).toBe(2); + await new Promise((r) => setTimeout(r, 0)); + expect(getClusterInflight()).toBe(0); + }); +}); diff --git a/packages/civitai-redis/src/__tests__/cluster-selfheal.test.ts b/packages/civitai-redis/src/__tests__/cluster-selfheal.test.ts new file mode 100644 index 0000000000..9ab1a1454a --- /dev/null +++ b/packages/civitai-redis/src/__tests__/cluster-selfheal.test.ts @@ -0,0 +1,451 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; +import { ClusterSelfHealWatchdog } from '../cluster-selfheal'; +import type { ClusterSelfHealConfig, ClusterSelfHealDeps } from '../cluster-selfheal'; + +// ClusterSelfHealWatchdog is the FIX #1 self-heal for the node-redis cluster inflight-leak +// wedge: when a pod's tracked cluster inflight stays PINNED above the threshold for a +// sustained window, force exactly ONE reconnect (subject to a cooldown), the only thing that +// clears the orphaned `_execute` promises short of a process restart. The watchdog is pure +// (no redis/prom imports) and driven by tick() against a fake clock + injected counter, so we +// can prove: a sustained breach fires exactly one reconnect; a transient spike does NOT; the +// cooldown bounds reconnect frequency; disabled is a no-op; the reconnect counter fires with +// the inflight-at-trigger value; a reconnect rejection doesn't wedge the watchdog. Mirrors +// command-deadline.test.ts / client.test.ts (each fix in this area regressed once — lock it). + +const DEFAULTS: ClusterSelfHealConfig = { + enabled: true, + inflightThreshold: 50, + sustainedMs: 20000, + cooldownMs: 60000, + // Default the deadline-hit trigger OFF in this harness so the legacy inflight-path tests + // below are unaffected; the deadline-trigger tests opt in explicitly. + deadlineHitThreshold: 0, + deadlineHitWindowMs: 20000, + // Default jitter OFF so every existing test fires the reconnect immediately (back-compat); + // the jitter tests opt in explicitly and inject a deterministic clock + delay. + reconnectJitterMs: 0, +}; + +/** Test harness: a controllable clock + inflight value + spy reconnect/onReconnect. */ +function makeHarness( + cfg: Partial = {}, + opts: { + reconnect?: () => Promise; + /** Fixed value [0,1) the watchdog's `random()` returns (for deterministic jitter). */ + random?: number; + /** + * When true, the injected `delay` does NOT auto-resolve — each call is parked and only + * settles when the test calls `releaseDelays()`. Lets us prove the reconnect is held during + * the jitter window and that a mid-delay tick single-flights out. + */ + manualDelay?: boolean; + } = {} +) { + let nowMs = 0; + let inflight = 0; + let deadlineHits = 0; + const reconnect = vi.fn(opts.reconnect ?? (() => Promise.resolve())); + const onReconnect = vi.fn(); + const resetDeadlineHits = vi.fn(() => { + deadlineHits = 0; + }); + const log = vi.fn(); + // Records every requested jitter delay; in manualDelay mode each is held until released. + const delayCalls: number[] = []; + const pendingResolvers: Array<() => void> = []; + const delay = vi.fn((ms: number) => { + delayCalls.push(ms); + if (!opts.manualDelay) return Promise.resolve(); + return new Promise((r) => pendingResolvers.push(r)); + }); + const deps: ClusterSelfHealDeps = { + getInflight: () => inflight, + getDeadlineHits: () => deadlineHits, + resetDeadlineHits, + reconnect, + now: () => nowMs, + log, + onReconnect, + delay, + random: () => opts.random ?? 0, + }; + const watchdog = new ClusterSelfHealWatchdog({ ...DEFAULTS, ...cfg }, deps); + return { + watchdog, + reconnect, + onReconnect, + resetDeadlineHits, + log, + delay, + delayCalls, + /** Resolve all parked manual-delay promises (the jitter windows elapse). */ + releaseDelays: () => { + const rs = pendingResolvers.splice(0); + rs.forEach((r) => r()); + }, + setInflight: (v: number) => (inflight = v), + setDeadlineHits: (v: number) => (deadlineHits = v), + advance: (ms: number) => (nowMs += ms), + setNow: (ms: number) => (nowMs = ms), + now: () => nowMs, + }; +} + +/** Flush all pending microtasks (the reconnect's then→catch→finally chain settles). */ +const flush = () => new Promise((r) => setTimeout(r, 0)); + +/** Drive ticks across `ms` at `step` granularity (simulates the watchdog interval). */ +function runFor(h: ReturnType, ms: number, step = 1000): number { + let fires = 0; + for (let t = 0; t < ms; t += step) { + if (h.watchdog.tick()) fires++; + h.advance(step); + } + return fires; +} + +describe('ClusterSelfHealWatchdog', () => { + beforeEach(() => vi.clearAllMocks()); + + it('fires exactly one reconnect when inflight stays pinned above the threshold for the sustained window', async () => { + const h = makeHarness(); + h.setInflight(500); // wedged: pinned well above 50 + + // Below the sustained window: no reconnect yet. + const firesEarly = runFor(h, DEFAULTS.sustainedMs, 1000); // ticks across exactly the window + expect(firesEarly).toBe(0); + expect(h.reconnect).not.toHaveBeenCalled(); + + // One more tick now that now-breachStart >= sustainedMs → trigger. + expect(h.watchdog.tick()).toBe(true); + expect(h.reconnect).toHaveBeenCalledTimes(1); + expect(h.onReconnect).toHaveBeenCalledTimes(1); + // onReconnect carries the inflight value at trigger time + which trigger fired (for the + // Prom counter label / Loki line). This is the legacy sustained-inflight path → 'inflight'. + expect(h.onReconnect).toHaveBeenCalledWith(500, 'inflight'); + + await flush(); // let the fire-and-forget reconnect settle + }); + + it('does NOT fire on a transient spike that drops back under the threshold before the window elapses', () => { + const h = makeHarness(); + h.setInflight(500); // spike up + // Spike lasts only ~half the window. + runFor(h, DEFAULTS.sustainedMs / 2, 1000); + expect(h.reconnect).not.toHaveBeenCalled(); + + // Drops back to healthy — the sustained timer must RESET. + h.setInflight(0); + h.watchdog.tick(); + expect(h.watchdog.getState().breachStartedAt).toBeNull(); + + // Spike again, but again only briefly: still no reconnect (timer restarted from here). + h.setInflight(500); + runFor(h, DEFAULTS.sustainedMs / 2, 1000); + expect(h.reconnect).not.toHaveBeenCalled(); + }); + + it('respects the cooldown — at most one reconnect per cooldown window even while inflight stays pinned', async () => { + const h = makeHarness(); + h.setInflight(500); // stays wedged the whole time + + // First trigger after the sustained window. + runFor(h, DEFAULTS.sustainedMs, 1000); + expect(h.watchdog.tick()).toBe(true); + expect(h.reconnect).toHaveBeenCalledTimes(1); + await flush(); // reconnect resolves → reconnecting=false + + // Keep ticking through the cooldown while still pinned: no second reconnect. + const firesDuringCooldown = runFor(h, DEFAULTS.cooldownMs, 1000); + expect(firesDuringCooldown).toBe(0); + expect(h.reconnect).toHaveBeenCalledTimes(1); + + // Past the cooldown while still wedged, exactly ONE more reconnect fires (the breach + // timer kept running through the cooldown, so it's already satisfied). Drive a generous + // window and assert the count, since the firing tick is consumed inside runFor. + const firesAfterCooldown = runFor(h, DEFAULTS.sustainedMs + 2000, 1000); + expect(firesAfterCooldown).toBe(1); + expect(h.reconnect).toHaveBeenCalledTimes(2); + await flush(); + }); + + it('is a no-op when disabled (kill switch): never reconnects, clears the breach timer', () => { + const h = makeHarness({ enabled: false }); + h.setInflight(100000); // extreme wedge + const fires = runFor(h, DEFAULTS.sustainedMs * 5, 1000); + expect(fires).toBe(0); + expect(h.reconnect).not.toHaveBeenCalled(); + expect(h.onReconnect).not.toHaveBeenCalled(); + expect(h.watchdog.getState().breachStartedAt).toBeNull(); + }); + + it('does not fire while inflight equals the threshold exactly (strictly-above only)', () => { + const h = makeHarness(); + h.setInflight(DEFAULTS.inflightThreshold); // == 50, not > 50 + const fires = runFor(h, DEFAULTS.sustainedMs * 2, 1000); + expect(fires).toBe(0); + expect(h.reconnect).not.toHaveBeenCalled(); + }); + + it('single-flights: does not start a second reconnect while one is in progress', async () => { + // A reconnect that resolves only when we let it, to hold reconnecting=true. + let release!: () => void; + const gate = new Promise((r) => (release = r)); + const h = makeHarness({}, { reconnect: () => gate }); + h.setInflight(500); + + runFor(h, DEFAULTS.sustainedMs, 1000); + expect(h.watchdog.tick()).toBe(true); // first trigger, reconnect pending + expect(h.reconnect).toHaveBeenCalledTimes(1); + + // While the reconnect is pending, further ticks (even past another window) must NOT + // start a second reconnect. + runFor(h, DEFAULTS.sustainedMs * 3, 1000); + expect(h.reconnect).toHaveBeenCalledTimes(1); + + release(); + await gate; + await flush(); + expect(h.watchdog.getState().reconnecting).toBe(false); + }); + + it('survives a rejecting reconnect: logs it, clears reconnecting, and re-arms after cooldown', async () => { + const err = new Error('disconnect failed'); + const h = makeHarness({}, { reconnect: () => Promise.reject(err) }); + h.setInflight(500); + + runFor(h, DEFAULTS.sustainedMs, 1000); + expect(h.watchdog.tick()).toBe(true); + expect(h.reconnect).toHaveBeenCalledTimes(1); + + // Let the rejected reconnect settle. + await flush(); + expect(h.watchdog.getState().reconnecting).toBe(false); + // The failure was logged, not thrown. + expect(h.log.mock.calls.some(([m]) => String(m).includes('reconnect failed'))).toBe(true); + + // After the cooldown it can try again (still wedged). The firing tick is consumed inside + // runFor, so assert on the reconnect count over a generous window. + runFor(h, DEFAULTS.cooldownMs, 1000); + const firesAgain = runFor(h, DEFAULTS.sustainedMs + 2000, 1000); + expect(firesAgain).toBe(1); + expect(h.reconnect).toHaveBeenCalledTimes(2); + await flush(); + }); + + // ── DEADLINE-HIT TRIGGER (the fix for the real-wave non-firing bug) ──────────────── + + it('REGRESSION: a sawtoothing inflight (deadline drains it every ~15s) never fires the inflight trigger', () => { + // Reproduces the live bug: the 15s command deadline mass-rejects parked commands, so + // inflight crashes below the threshold within each 20s sustained window → breachStartedAt + // keeps resetting → the inflight trigger can NEVER accumulate. Deadline trigger OFF here to + // isolate the inflight path. Simulate the sawtooth: ~14s pinned high, then a 1s crash to 0. + const h = makeHarness({ deadlineHitThreshold: 0 }); + let fires = 0; + for (let cycle = 0; cycle < 10; cycle++) { + h.setInflight(200); + for (let t = 0; t < 14000; t += 1000) { + if (h.watchdog.tick()) fires++; + h.advance(1000); + } + // Deadline batch rejects → inflight crashes below threshold for one sample. + h.setInflight(0); + if (h.watchdog.tick()) fires++; + h.advance(1000); + } + // ~150s of a real half-open wedge, zero reconnects — exactly the observed prod behavior. + expect(fires).toBe(0); + expect(h.reconnect).not.toHaveBeenCalled(); + }); + + it('FIX: the deadline-hit trigger fires on the SAME sawtoothing wedge, regardless of inflight dips', async () => { + // Same sawtooth, but now the deadline-hit trigger is armed (the drains ARE the hits, so the + // hit count stays high even while inflight dips). It must fire on the very first tick once + // the hit count is at/above threshold, without needing any inflight continuity. + const h = makeHarness({ deadlineHitThreshold: 10, deadlineHitWindowMs: 20000 }); + h.setInflight(0); // inflight can be ANYTHING — deadline trigger is independent of it + h.setDeadlineHits(25); // 25 deadline timeouts in the last 20s window >= 10 + + expect(h.watchdog.tick()).toBe(true); + expect(h.reconnect).toHaveBeenCalledTimes(1); + expect(h.onReconnect).toHaveBeenCalledTimes(1); + // onReconnect is told WHICH trigger fired so client.ts emits the `trigger="deadline"` + // metric label — the series we watch at the next prod wave to confirm THIS path fired. + expect(h.onReconnect).toHaveBeenCalledWith(expect.any(Number), 'deadline'); + // The window is cleared on trigger so the same pre-heal hits can't immediately re-fire. + expect(h.resetDeadlineHits).toHaveBeenCalledTimes(1); + await flush(); + }); + + it('does NOT fire the deadline trigger below the hit threshold (a one-off transient slow command)', () => { + const h = makeHarness({ deadlineHitThreshold: 10, deadlineHitWindowMs: 20000 }); + h.setInflight(0); + h.setDeadlineHits(9); // one short of the threshold + const fires = runFor(h, DEFAULTS.sustainedMs * 3, 1000); + expect(fires).toBe(0); + expect(h.reconnect).not.toHaveBeenCalled(); + }); + + it('deadline trigger respects the cooldown (one reconnect per cooldown even while hits stay high)', async () => { + const h = makeHarness({ deadlineHitThreshold: 10, deadlineHitWindowMs: 20000 }); + h.setInflight(0); + h.setDeadlineHits(100); // stays wedged + + expect(h.watchdog.tick()).toBe(true); + expect(h.reconnect).toHaveBeenCalledTimes(1); + await flush(); + + // resetDeadlineHits zeroed the window; the wedge keeps producing hits, re-arming it. + h.setDeadlineHits(100); + // Within the cooldown: no second reconnect. + const firesDuringCooldown = runFor(h, DEFAULTS.cooldownMs, 1000); + expect(firesDuringCooldown).toBe(0); + expect(h.reconnect).toHaveBeenCalledTimes(1); + + // Past the cooldown, still wedged → exactly one more. + h.setDeadlineHits(100); + const firesAfter = runFor(h, 3000, 1000); + expect(firesAfter).toBe(1); + expect(h.reconnect).toHaveBeenCalledTimes(2); + await flush(); + }); + + it('deadline trigger is inert when its threshold is 0 (falls back to the inflight path only)', () => { + const h = makeHarness({ deadlineHitThreshold: 0 }); + h.setInflight(0); + h.setDeadlineHits(100000); // enormous, but the trigger is disabled + const fires = runFor(h, DEFAULTS.sustainedMs * 3, 1000); + expect(fires).toBe(0); + expect(h.reconnect).not.toHaveBeenCalled(); + }); + + it('deadline trigger is inert when getDeadlineHits dep is omitted (back-compat)', () => { + // A caller (or older test) that doesn't supply getDeadlineHits must behave as before. + let nowMs = 0; + let inflight = 0; + const reconnect = vi.fn(() => Promise.resolve()); + const deps: ClusterSelfHealDeps = { + getInflight: () => inflight, + reconnect, + now: () => nowMs, + log: vi.fn(), + onReconnect: vi.fn(), + }; + const watchdog = new ClusterSelfHealWatchdog( + { ...DEFAULTS, deadlineHitThreshold: 10 }, + deps + ); + inflight = 0; // inflight path never trips + for (let t = 0; t < DEFAULTS.sustainedMs * 3; t += 1000) { + watchdog.tick(); + nowMs += 1000; + } + expect(reconnect).not.toHaveBeenCalled(); + }); + + it('still fires the inflight trigger when inflight genuinely stays pinned (no deadline drain)', async () => { + // Belt-and-suspenders: a wedge that leaks inflight WITHOUT deadline-rejecting (e.g. deadline + // disabled) must still be caught by the legacy continuous-breach path. + const h = makeHarness({ deadlineHitThreshold: 10 }); + h.setInflight(200); // pinned, never dips + h.setDeadlineHits(0); // no deadline hits at all + runFor(h, DEFAULTS.sustainedMs, 1000); + expect(h.watchdog.tick()).toBe(true); + expect(h.reconnect).toHaveBeenCalledTimes(1); + // Reported as the legacy 'inflight' trigger (no deadline hits) → metric label distinguishes it. + expect(h.onReconnect).toHaveBeenCalledWith(expect.any(Number), 'inflight'); + await flush(); + }); + + // ── PER-POD RECONNECT JITTER (fleet-stampede brake) ──────────────────────────────── + + it('JITTER: delays the actual reconnect by a random [0, jitterMs) before calling reconnect()', async () => { + // random()=0.5, jitterMs=1000 → floor(0.5 * 1000) = 500ms delay. With manualDelay the + // reconnect stays UN-called until the jitter window is released, proving the delay precedes + // the destroy()+connect() (the fleet-stampede smear). + const h = makeHarness( + { deadlineHitThreshold: 10, deadlineHitWindowMs: 20000, reconnectJitterMs: 1000 }, + { random: 0.5, manualDelay: true } + ); + h.setDeadlineHits(25); // trip the deadline trigger + + expect(h.watchdog.tick()).toBe(true); + // The trigger DECISION happened (guards taken, onReconnect emitted) but the reconnect is + // still parked behind the jitter delay. + expect(h.onReconnect).toHaveBeenCalledTimes(1); + expect(h.delay).toHaveBeenCalledTimes(1); + expect(h.delayCalls[0]).toBe(500); // floor(0.5 * 1000) + expect(h.delayCalls[0]).toBeLessThan(1000); // strictly within [0, jitterMs) + expect(h.reconnect).not.toHaveBeenCalled(); // NOT yet — held by the jitter window + // Single-flight + cooldown guards are already in place during the delay. + expect(h.watchdog.getState().reconnecting).toBe(true); + expect(h.watchdog.getState().lastReconnectAt).toBe(0); // cooldown clock starts at decision time + + // Jitter window elapses → the reconnect finally fires, then reconnecting clears. + h.releaseDelays(); + await flush(); + expect(h.reconnect).toHaveBeenCalledTimes(1); + expect(h.watchdog.getState().reconnecting).toBe(false); + }); + + it('JITTER=0: fires the reconnect immediately (back-compat — no delay path at all)', async () => { + // jitterMs=0 → delay() is never called; reconnect() is invoked SYNCHRONOUSLY within tick(), + // exactly the prior behavior (the existing suite relies on this synchronous call). + const h = makeHarness({ deadlineHitThreshold: 10, deadlineHitWindowMs: 20000, reconnectJitterMs: 0 }); + h.setDeadlineHits(25); + + expect(h.watchdog.tick()).toBe(true); + expect(h.delay).not.toHaveBeenCalled(); // no jitter → no delay step + expect(h.reconnect).toHaveBeenCalledTimes(1); // called synchronously, not deferred + await flush(); + expect(h.watchdog.getState().reconnecting).toBe(false); + }); + + it('JITTER single-flight: a second tick DURING the jitter delay does NOT schedule a second reconnect', async () => { + // The single-flight guard (reconnecting=true) is taken at DECISION time, before the jitter + // delay — so a watchdog tick that fires while the first reconnect is still parked behind its + // jitter window must be a no-op. This is the correlated-fleet correctness property: the delay + // must NOT open a window where the wedge re-counts and double-fires. + const h = makeHarness( + { deadlineHitThreshold: 10, deadlineHitWindowMs: 20000, reconnectJitterMs: 1000 }, + { random: 0.9, manualDelay: true } // floor(0.9*1000)=900ms, held open + ); + h.setDeadlineHits(100); // stays wedged across both ticks + + expect(h.watchdog.tick()).toBe(true); // first trigger → parked behind jitter + expect(h.delay).toHaveBeenCalledTimes(1); + expect(h.reconnect).not.toHaveBeenCalled(); + + // Tick again mid-jitter (still wedged). Must single-flight out: no second delay, no second + // reconnect, no second onReconnect/resetDeadlineHits. + h.advance(500); // partway through the 900ms jitter window + expect(h.watchdog.tick()).toBe(false); + expect(h.delay).toHaveBeenCalledTimes(1); + expect(h.onReconnect).toHaveBeenCalledTimes(1); + expect(h.resetDeadlineHits).toHaveBeenCalledTimes(1); + expect(h.reconnect).not.toHaveBeenCalled(); + + // Release the (single) jitter window → exactly ONE reconnect total. + h.releaseDelays(); + await flush(); + expect(h.reconnect).toHaveBeenCalledTimes(1); + expect(h.watchdog.getState().reconnecting).toBe(false); + }); + + it('tick() never throws even if the onReconnect hook throws, and still reconnects', async () => { + const h = makeHarness(); + h.onReconnect.mockImplementation(() => { + throw new Error('counter boom'); + }); + h.setInflight(500); + runFor(h, DEFAULTS.sustainedMs, 1000); + // A throwing onReconnect (a broken Prom counter) must NOT abort the reconnect or wedge + // the watchdog: tick still returns true, the reconnect still fires, and reconnecting + // clears once it settles. + expect(() => h.watchdog.tick()).not.toThrow(); + expect(h.reconnect).toHaveBeenCalledTimes(1); + await flush(); + expect(h.watchdog.getState().reconnecting).toBe(false); + expect(h.watchdog.getState().breachStartedAt).toBeNull(); + }); +}); diff --git a/packages/civitai-redis/src/__tests__/packed-compression.test.ts b/packages/civitai-redis/src/__tests__/packed-compression.test.ts new file mode 100644 index 0000000000..bb549bd50e --- /dev/null +++ b/packages/civitai-redis/src/__tests__/packed-compression.test.ts @@ -0,0 +1,95 @@ +import { pack, unpack } from 'msgpackr'; +import { describe, expect, it } from 'vitest'; +import { + PACKED_BROTLI_SENTINEL, + compressPacked, + decompressPacked, +} from '../packed-compression'; + +/** + * Round-trip + back-compat coverage for the opt-in brotli compression of `redis.packed` + * values (the tensor-metadata whale fix). compressPacked/decompressPacked are ASYNC + * (libuv-threadpool brotli, so the codec never blocks the event loop). The contract: + * + * - compress (new write): pack → brotli → sentinel-prefix; decompress strips the + * sentinel + inflates back to the EXACT original packed bytes → unpack === original. + * - back-compat (legacy read): a raw-msgpack Buffer with NO sentinel must pass through + * decompressPacked untouched so it still unpacks correctly (the ~220k existing keys). + */ +describe('packed brotli compression', () => { + // Mirrors the on-the-wire shape every fetchThroughCache value has: the `{ data, cachedAt }` + // wrapper object, here wrapping a tensor-metadata-like analysis with repetitive names. + const sampleAnalysis = { + data: { + format: 'SafeTensor', + tensorCount: 3, + totalTensorBytes: 123456, + dtypeCounts: [{ dtype: 'F16', count: 3, bytes: 123456 }], + largestTensor: { name: 'model.diffusion_model.blocks.0.weight', shape: [320, 320], dtype: 'F16', sizeBytes: 50000 }, + vramEstimate: null, + tensors: Array.from({ length: 200 }, (_, i) => ({ + name: `model.diffusion_model.blocks.${i}.attn.to_q.weight`, + shape: [320, 320], + dtype: 'F16', + sizeBytes: 204800, + })), + }, + cachedAt: 1_700_000_000_000, + }; + + it('round-trips: pack → compress → decompress → unpack equals the original', async () => { + const packed = Buffer.from(pack(sampleAnalysis)); + const compressed = await compressPacked(packed); + + // Sentinel-tagged and actually smaller (highly repetitive payload). + expect(compressed[0]).toBe(PACKED_BROTLI_SENTINEL); + expect(compressed.length).toBeLessThan(packed.length); + + const restored = await decompressPacked(compressed); + expect(Buffer.compare(restored, packed)).toBe(0); // exact bytes + expect(unpack(restored)).toEqual(sampleAnalysis); // exact value + }); + + it('back-compat: a legacy raw-msgpack buffer (no sentinel) decodes unchanged', async () => { + const legacy = Buffer.from(pack(sampleAnalysis)); // simulates an existing uncompressed key + + // Sanity: legacy wrapper objects start with a msgpack MAP marker, never the sentinel. + expect(legacy[0]).not.toBe(PACKED_BROTLI_SENTINEL); + const marker = legacy[0]; + const isFixmap = marker >= 0x80 && marker <= 0x8f; + expect(isFixmap || marker === 0xde || marker === 0xdf).toBe(true); + + const passthrough = await decompressPacked(legacy); + expect(Buffer.compare(passthrough, legacy)).toBe(0); // untouched + expect(unpack(passthrough)).toEqual(sampleAnalysis); + }); + + it('decompressPacked is a no-op on an empty buffer (defensive)', async () => { + const empty = Buffer.alloc(0); + expect(await decompressPacked(empty)).toBe(empty); + }); + + it('compresses a large repetitive blob substantially (the whale property)', async () => { + const big = Buffer.from( + pack({ + data: { tensors: Array.from({ length: 2000 }, (_, i) => `transformer.h.${i}.mlp.c_fc.weight`) }, + cachedAt: 0, + }) + ); + const compressed = await compressPacked(big); + // Not asserting the measured 64.9x (codec/version-dependent), just that it's a big win. + expect(compressed.length).toBeLessThan(big.length / 4); + expect(unpack(await decompressPacked(compressed))).toEqual(unpack(big)); + }); + + // The brotli sentinel (0x01) collides with the msgpack encoding of the bare integer 1 + // (`pack(1) === <01>`). This is harmless ONLY because decompression is confined to the + // compress-aware read path, which only ever sees the `{ data, cachedAt }` wrapper + // (first byte = MAP marker). Pin the collision so the confinement rationale stays true: + // a bare scalar MUST NOT be routed through compression on a shared path. + it('documents the sentinel↔bare-scalar collision that justifies confinement', () => { + expect(Buffer.from(pack(1))[0]).toBe(PACKED_BROTLI_SENTINEL); // pack(1) === <01> + // pack(0) is 0x00 — distinct from the sentinel, but still must never be decompressed. + expect(Buffer.from(pack(0))[0]).not.toBe(PACKED_BROTLI_SENTINEL); + }); +}); diff --git a/packages/civitai-redis/src/__tests__/packed-decode-paths.test.ts b/packages/civitai-redis/src/__tests__/packed-decode-paths.test.ts new file mode 100644 index 0000000000..8d095553af --- /dev/null +++ b/packages/civitai-redis/src/__tests__/packed-decode-paths.test.ts @@ -0,0 +1,122 @@ +import { pack, unpack } from 'msgpackr'; +import { describe, expect, it } from 'vitest'; +import { + PACKED_BROTLI_SENTINEL, + compressPacked, + decompressPacked, +} from '../packed-compression'; + +/** + * Regression guard for the hardened decode design (PR #2649 review Fix 1). + * + * `redis.packed` has TWO decode paths in src/server/redis/client.ts: + * + * GENERAL (safeUnpack, used by get/mGet/sMembers/sPop/hGet/hGetAll/hmGet across ~30 + * cache writers): plain `unpack(value)` — NEVER decompresses. + * COMPRESS-AWARE (safeUnpackCompressed, used ONLY by get(key, { compress: true }), i.e. + * fetchThroughCache's compressed callers): sentinel-detect → brotli-decompress + * → unpack, back-compat with legacy uncompressed wrappers. + * + * The brotli sentinel (0x01) collides with `pack(1)` (=== <01>). Routing decompression + * through the SHARED general path would mis-read a bare `1` (a count/flag/scalar tRPC + * result) as a compressed payload → throw → evict → permanent cache miss. Confining + * decompression to the compress-aware path makes the collision harmless: that path only + * ever sees the `{ data, cachedAt }` wrapper (first byte = MAP marker, never 0x01). + * + * These tests model BOTH paths at the exact decode boundary the wrappers use (the same + * `unpack` / `decompressPacked` calls as client.ts), without booting the full client + * module (which opens TCP sockets at import). + */ + +// Mirrors safeUnpack in client.ts verbatim (the GENERAL path: plain unpack, no decompress). +function safeUnpack(value: Buffer): T | null { + try { + return unpack(value) as T; + } catch { + return null; + } +} + +// Mirrors safeUnpackCompressed in client.ts (the COMPRESS-AWARE path: sentinel-detect → +// decompress → unpack, evict-on-throw → null). +async function safeUnpackCompressed(value: Buffer): Promise { + try { + return unpack(await decompressPacked(value)) as T; + } catch { + return null; + } +} + +describe('packed decode paths (Fix 1 — confined decompression)', () => { + describe('GENERAL path is collision-safe (the Fix 1 regression guard)', () => { + it('a bare integer 1 round-trips as 1, NOT mistaken for a compressed payload', () => { + const stored = Buffer.from(pack(1)); // === <01>, collides with the brotli sentinel + expect(stored[0]).toBe(PACKED_BROTLI_SENTINEL); + + // The general decode path must return the integer 1, never throw / null. + expect(safeUnpack(stored)).toBe(1); + }); + + it('a bare integer 0 round-trips as 0 on the general path', () => { + const stored = Buffer.from(pack(0)); + expect(safeUnpack(stored)).toBe(0); + }); + + it('the general path never invokes brotli-decompress for a sentinel-byte scalar', () => { + // Proven structurally: safeUnpack calls unpack(value) directly with no decompress + // step. A bare `1` (<01>) is NOT a valid brotli stream — if it WERE routed through + // decompressPacked it would throw. Confirm the general path returns 1 regardless. + const one = Buffer.from(pack(1)); + expect(safeUnpack(one)).toBe(1); + // And confirm that feeding the same byte to the compress-aware decompress WOULD + // fail (so the only thing keeping bare-1 readable is the path confinement). + return expect(decompressPacked(one)).rejects.toThrow(); + }); + }); + + describe('COMPRESS-AWARE path: compressed round-trip + legacy back-compat', () => { + const wrapper = { + data: { + format: 'SafeTensor', + tensorCount: 2, + tensors: Array.from({ length: 50 }, (_, i) => ({ name: `blocks.${i}.weight` })), + }, + cachedAt: 1_700_000_000_000, + }; + + it('round-trips a compressed wrapper', async () => { + const compressed = await compressPacked(Buffer.from(pack(wrapper))); + expect(compressed[0]).toBe(PACKED_BROTLI_SENTINEL); + expect(await safeUnpackCompressed(compressed)).toEqual(wrapper); + }); + + it('reads a LEGACY uncompressed wrapper (no sentinel) — back-compat', async () => { + const legacy = Buffer.from(pack(wrapper)); // pre-compression key + expect(legacy[0]).not.toBe(PACKED_BROTLI_SENTINEL); // MAP marker + expect(await safeUnpackCompressed(legacy)).toEqual(wrapper); + }); + + it('a corrupt/garbage value is treated as a cache miss (null), preserving fail-open', async () => { + const garbage = Buffer.concat([Buffer.from([PACKED_BROTLI_SENTINEL]), Buffer.from('not-brotli')]); + expect(await safeUnpackCompressed(garbage)).toBeNull(); + }); + }); + + /** + * MIXED-FLEET CANARY NOTE (expected/safe, not a bug): + * + * During a rollout, an OLD pod (no compress-aware read) can read a NEW compressed + * tensor-metadata value written by a NEW pod. The old pod's safeUnpack does plain + * `unpack(<01...>)` → msgpack throws on the brotli stream → safeUnpack evicts the key + * and returns null (cache miss) → the request refetches uncompressed from origin and + * repopulates. Net effect: a few transient cache misses while the fleet converges, NO + * data corruption and NO 500s (the read is fail-open). This is the intended back-compat + * behavior of the sentinel design, exercised here for documentation. + */ + it('mixed-fleet: an OLD pod (general path) treats a NEW compressed value as a miss, not a crash', async () => { + const compressed = await compressPacked(Buffer.from(pack({ data: { x: 1 }, cachedAt: 0 }))); + // The OLD pod uses the general path (no decompress) → unpack of the brotli stream throws + // → safeUnpack returns null (caller evicts + refetches). No throw escapes, no corruption. + expect(safeUnpack(compressed)).toBeNull(); + }); +}); diff --git a/packages/civitai-redis/src/client.ts b/packages/civitai-redis/src/client.ts index 8973d95c6d..f9d3fd967a 100644 --- a/packages/civitai-redis/src/client.ts +++ b/packages/civitai-redis/src/client.ts @@ -6,6 +6,20 @@ import { RESP_TYPES } from 'redis'; import slugify from 'slugify'; import { loadRedisEnv, type RedisConfig } from './env'; import { withCommandDeadline } from './deadline'; +import { ClusterSelfHealWatchdog } from './cluster-selfheal'; +import type { ClusterSelfHealTrigger } from './cluster-selfheal'; +import { + decClusterInflight, + getClusterInflight, + incClusterInflight, + resetClusterInflight, +} from './cluster-inflight'; +import { + countClusterDeadlineHits, + recordClusterDeadlineHit, + resetClusterDeadlineHits, +} from './cluster-deadline-hits'; +import { compressPacked, decompressPacked } from './packed-compression'; export type { RedisConfig } from './env'; export type RedisLogFn = (message: string, ...args: unknown[]) => void; @@ -93,10 +107,22 @@ interface CustomRedisClient cursor?: string; }): AsyncGenerator; packed: { - get(key: K): Promise; + // `packedOptions.compress` opts the READ into the compress-aware decode path + // (sentinel-detect + brotli-decompress), SYMMETRIC with set's compress. Only enable + // for keys written with compress — see the SENTINEL SCOPE note in packed-compression. + get(key: K, packedOptions?: { compress?: boolean }): Promise; // Wrapped to avoid CROSSSLOT errors - fetches keys individually with Promise.all mGet(keys: K[]): Promise<(T | null)[]>; - set(key: K, value: T, setOptions?: SetOptions): Promise; + // `packedOptions.compress` opts the value into brotli compression at rest (sentinel- + // tagged; reads decode both compressed and legacy-uncompressed values transparently). + // Only safe for callers that store wrapper objects, never bare scalars — see the + // SENTINEL SAFETY note in the packed implementation. + set( + key: K, + value: T, + setOptions?: SetOptions, + packedOptions?: { compress?: boolean } + ): Promise; // mSet still disabled - sets are more complex with different argument formats // mSet(records: Record, setOptions?: SetOptions): Promise; setNX(key: K, value: T): Promise; @@ -203,6 +229,14 @@ let isEnhancedFailoverEnabled: RedisFailoverResolver = async () => false; // Track topology refresh intervals for cleanup const clusterRefreshIntervals = new Map>(); +// In-process count of in-flight CLUSTER commands lives in `./cluster-inflight` so EVERY +// mutation goes through a single floored helper (the counter can never go negative — see +// FIX #2 there). instrumentCommands inc/decs it in lockstep with the +// redis_commands_inflight{client="cluster"} gauge — the same signal the gauge exposes, read +// locally (the prom-client Gauge doesn't cheaply surface its value) so the self-heal watchdog +// (FIX #1) can sample it without a prom dependency. (The sys client uses 'sys' and is NOT +// tracked — the watchdog only ever acts on the cluster client.) + /** * Trigger topology rediscovery on a cluster client. * Uses internal _slots.rediscover() method - this is undocumented but stable. @@ -240,6 +274,200 @@ function triggerTopologyRediscovery(clusterClient: any, reason: string) { } } +// Hold the watchdog interval so it can be cleared on client close (mirrors +// clusterRefreshIntervals). Module-scoped: one cluster client per process. +let clusterSelfHealInterval: ReturnType | undefined; + +/** + * FIX #2: decide whether the periodic `_slots.rediscover()` should be scheduled, from the + * REDIS_CLUSTER_REFRESH_INTERVAL value. Pure (no side effects) so it can be unit-tested + * without booting the cluster client. + * + * Contract: + * - value > 0 → { enabled: true, intervalMs: value } (a LARGER value lengthens it) + * - value <= 0 → { enabled: false, intervalMs: 0 } (0 is the DISABLE SENTINEL) + * - NaN / non-finite (mis-set env) → treated as the default 30000 so a typo can't silently + * disable the refresh (the schema coerces, but be defensive). + * + * Default behavior is unchanged when the env is unset: server-schema defaults + * REDIS_CLUSTER_REFRESH_INTERVAL to 30000, so this returns { enabled: true, intervalMs: 30000 }. + */ +export function resolvePeriodicRefresh(refreshIntervalMs: number): { + enabled: boolean; + intervalMs: number; +} { + if (!Number.isFinite(refreshIntervalMs)) return { enabled: true, intervalMs: 30000 }; + if (refreshIntervalMs > 0) return { enabled: true, intervalMs: refreshIntervalMs }; + return { enabled: false, intervalMs: 0 }; // 0 (or negative) = disable sentinel +} + +/** + * Force a FULL reconnect of a node-redis cluster client (FIX #1). The teardown must REJECT + * the in-flight commands IMMEDIATELY (not drain them) and connect() rebuilds the node + * connections + re-runs slot discovery — which is the ONLY way (short of a process restart) + * to clear the orphaned `_execute` promises that the inflight-leak wedge accumulates. + * + * WHY `destroy()`, NOT `close()` (FIX #1 — verified against @redis/client@5.8.3): + * - cluster `close()` → `#destroy(c => c.close())` → `Promise.allSettled(perNodeClose)`, and + * per-node `RedisClient.close()` "waits for pending commands": it only resolves once the + * node's reply `#queue.isEmpty()`. The whole reason we're reconnecting is that commands + * are ORPHANED and never get a reply → the queue never empties → `close()` can HANG. + * Because the watchdog single-flights on `reconnecting=true` until reconnect settles, a + * hung `close()` pins `reconnecting=true` forever — the watchdog would be permanently dead + * after its first trigger, failing in exactly the scenario it exists for. + * - cluster `destroy()` → per-node `RedisClient.destroy()` = + * `#queue.flushAll(new DisconnectsClientError())` + `socket.destroy()`: it REJECTS every + * in-flight command immediately and tears the sockets down synchronously. That's what we + * want. (`disconnect()` is the legacy alias and also rejects immediately, but `destroy()` + * is the documented v5 method.) + * + * Each rejected in-flight command then runs its done() closure → decClusterInflight() (floored + * at 0, FIX #2), so the counter drains back to ~0 and the watchdog can re-arm. We also reset + * it up front so the watchdog's sampled value snaps clean at the trigger instant. + * + * Because `destroy()` is SYNCHRONOUS and does NOT go through the wrapped `close()` that clears + * the topology-refresh interval, this function explicitly clears that interval before tearing + * down (FIX #3) so it isn't leaked, then re-establishes failover after reconnect so the new + * connection gets exactly one fresh periodic-rediscover interval. + * + * Any teardown error is non-fatal: connect() is still attempted, and the watchdog re-arms + * after the cooldown regardless. + * + * @param reSetupFailover re-runs setupEnhancedFailover() after connect() resolves, so the + * periodic `_slots.rediscover()` interval (and node-error/disconnect rediscovery) is + * re-established post-heal exactly once (FIX #3). Injected by getBaseClient (the closure + * lives there); optional so unit tests can drive the function without it. + */ +async function forceClusterReconnect( + clusterClient: any, + reSetupFailover?: () => Promise +): Promise { + // FIX #3: clear the existing topology-refresh interval BEFORE the destroy(). destroy() + // bypasses the wrapped close() that would otherwise clear it, so without this the old + // interval leaks and reSetupFailover() below would schedule a SECOND one (net: two + // intervals firing rediscover on one client). Clearing here + re-creating in + // reSetupFailover keeps it at exactly one. + clearClusterRefreshInterval('cache'); + + try { + if (typeof clusterClient.destroy === 'function') { + // destroy() rejects in-flight commands immediately (flushAll(DisconnectsClientError)) + // and tears sockets down synchronously — does NOT wait for the wedged queue to drain. + clusterClient.destroy(); + } else if (typeof clusterClient.disconnect === 'function') { + // Legacy alias for destroy() — also rejects in-flight immediately. + await clusterClient.disconnect(); + } else if (typeof clusterClient.close === 'function') { + // Last resort only: close() can HANG on the wedged queue (see WHY above). Bound it so + // a hang can't pin reconnecting=true forever; fall through to connect() regardless. + await Promise.race([ + clusterClient.close(), + new Promise((resolve) => setTimeout(resolve, 2000)), + ]); + } + } catch (err) { + log( + `Cluster self-heal: teardown threw (continuing to reconnect): ${ + err instanceof Error ? err.message : String(err) + }` + ); + } + // Reset the local inflight counter up front — destroy() rejected every in-flight command, + // so their done() closures fire as the rejections settle and EACH floored-decrements toward + // 0 (FIX #2 — the floor is what stops a post-heal burst of rejects driving it negative). The + // reset just snaps the watchdog's sampled value clean at the trigger instant rather than + // letting it decay over the next few ticks. + resetClusterInflight(); + + await clusterClient.connect(); + + // FIX #3: re-establish enhanced failover (incl. the periodic-rediscover interval) on the + // freshly-connected client. setupEnhancedFailover clears any prior interval for this client + // kind first, so combined with the clear above this yields exactly ONE interval (honoring + // the REDIS_CLUSTER_REFRESH_INTERVAL disable sentinel). Non-fatal if it throws. + if (reSetupFailover) { + try { + await reSetupFailover(); + } catch (err) { + log( + `Cluster self-heal: re-setup failover after reconnect failed: ${ + err instanceof Error ? err.message : String(err) + }` + ); + } + } +} + +/** Clear (and forget) the topology-refresh interval for a client kind, if scheduled. */ +function clearClusterRefreshInterval(type: 'cache' | 'system'): void { + const interval = clusterRefreshIntervals.get(type); + if (interval) { + clearInterval(interval); + clusterRefreshIntervals.delete(type); + } +} + +/** + * Start the cluster-client self-heal watchdog (FIX #1). Samples the local cluster inflight + * counter on REDIS_CLUSTER_SELFHEAL_CHECK_INTERVAL_MS and forces a full reconnect when it + * stays pinned above the threshold for the sustained window (subject to the cooldown). All + * thresholds are env-tunable; REDIS_CLUSTER_SELFHEAL_ENABLED=false disables it entirely. + * Cluster client ONLY. Exposed (returns the watchdog) so a unit test can construct one with + * injected deps without booting the module-level interval here. + */ +function startClusterSelfHeal( + clusterClient: any, + reSetupFailover?: () => Promise +): ClusterSelfHealWatchdog { + const watchdog = new ClusterSelfHealWatchdog( + { + enabled: config.clusterSelfHealEnabled, + inflightThreshold: config.clusterSelfHealInflightThreshold, + sustainedMs: config.clusterSelfHealSustainedMs, + cooldownMs: config.clusterSelfHealCooldownMs, + deadlineHitThreshold: config.clusterSelfHealDeadlineHitThreshold, + deadlineHitWindowMs: config.clusterSelfHealDeadlineHitWindowMs, + reconnectJitterMs: config.clusterSelfHealReconnectJitterMs, + }, + { + getInflight: () => getClusterInflight(), + getDeadlineHits: (windowMs: number) => countClusterDeadlineHits(windowMs), + resetDeadlineHits: () => resetClusterDeadlineHits(), + reconnect: () => forceClusterReconnect(clusterClient, reSetupFailover), + log, + onReconnect: (inflightAtTrigger: number, trigger: ClusterSelfHealTrigger) => { + // Labeled by trigger so a rising `trigger="deadline"` series at the next prod wave is the + // direct confirmation that the fix's new path fired (the one the inflight path could + // never reach). See cluster-selfheal.ts and prom/client.ts. + getRedisMetrics()?.redisSelfHealReconnectCounter?.inc({ trigger }); + log( + `Cluster self-heal RECONNECT fired [trigger=${trigger}, inflight=${inflightAtTrigger}, inflightThreshold=${config.clusterSelfHealInflightThreshold}, sustainedMs=${config.clusterSelfHealSustainedMs}, deadlineHitThreshold=${config.clusterSelfHealDeadlineHitThreshold}, deadlineHitWindowMs=${config.clusterSelfHealDeadlineHitWindowMs}]` + ); + }, + } + ); + + if (!config.clusterSelfHealEnabled) { + log('Cluster self-heal watchdog DISABLED (REDIS_CLUSTER_SELFHEAL_ENABLED=false)'); + return watchdog; + } + + const interval = Math.max(250, config.clusterSelfHealCheckIntervalMs); + clusterSelfHealInterval = setInterval(() => { + try { + watchdog.tick(); + } catch (err) { + log(`Cluster self-heal tick error: ${err instanceof Error ? err.message : String(err)}`); + } + }, interval); + // Don't keep the event loop alive for the watchdog on a graceful process exit. + clusterSelfHealInterval.unref?.(); + log( + `Cluster self-heal watchdog ENABLED [inflightThreshold=${config.clusterSelfHealInflightThreshold}, sustainedMs=${config.clusterSelfHealSustainedMs}, deadlineHitThreshold=${config.clusterSelfHealDeadlineHitThreshold}, deadlineHitWindowMs=${config.clusterSelfHealDeadlineHitWindowMs}, reconnectJitterMs=${config.clusterSelfHealReconnectJitterMs}, cooldownMs=${config.clusterSelfHealCooldownMs}, checkMs=${interval}]` + ); + return watchdog; +} + /** * Parse cluster node URLs from environment variable. * Falls back to single URL if REDIS_CLUSTER_NODES is not set. @@ -282,6 +510,12 @@ type RedisMetricsBridge = { sysredisSentinelClientErrorsCounter: { labels: (labels: Record) => { inc: () => void }; }; + // FIX #1 self-heal reconnect counter (labeled by `trigger`: 'deadline' | 'inflight') + FIX #3 + // metric-write fail-soft counter. Read via the globalThis bridge so this client-bundle- + // reachable module avoids a static prom-client import (which drags in fs/cluster). Both may be + // undefined until prom/client loads server-side; callers null-check. + redisSelfHealReconnectCounter?: { inc: (labels: { trigger: string }) => void }; + redisMetricWriteFailSoftCounter?: { labels: (labels: { op: string }) => { inc: () => void } }; }; function getRedisMetrics(): RedisMetricsBridge | undefined { @@ -356,8 +590,19 @@ function instrumentCommands(client: any, clientLabel: 'cluster' | 'sys') { // prom module loads mid-flight (a dec without its inc would drive the gauge negative). const metrics = getRedisMetrics(); metrics?.redisCommandsInflight.inc(labels); + // Mirror the gauge into the local counter for the self-heal watchdog (cluster only). + if (clientLabel === 'cluster') incClusterInflight(); const start = Date.now(); + let dec = false; // per-closure guard so a double-settle (shouldn't happen) can't double-dec const done = () => { + if (clientLabel === 'cluster' && !dec) { + // FLOORED decrement (decClusterInflight clamps at 0). The per-closure `dec` guard above + // only stops THIS closure decrementing twice — it does NOT stop N distinct rejected + // closures (e.g. the burst destroy() rejects after a heal) decrementing past 0. The + // floor is what keeps the counter accurate post-heal so the watchdog can re-arm (FIX #2). + decClusterInflight(); + dec = true; + } metrics?.redisCommandsInflight.dec(labels); metrics?.redisCommandDuration.observe(labels, (Date.now() - start) / 1000); }; @@ -370,7 +615,7 @@ function instrumentCommands(client: any, clientLabel: 'cluster' | 'sys') { throw err; } const settled = wrapWithDeadline - ? withCommandDeadline(Promise.resolve(result), deadlineMs) + ? withCommandDeadline(Promise.resolve(result), deadlineMs, recordClusterDeadlineHit) : Promise.resolve(result); return settled.finally(done); }; @@ -583,8 +828,25 @@ function getBaseClient(type: 'cache' | 'system') { log('Enhanced cluster failover handling is ENABLED'); - const refreshInterval = config.clusterRefreshInterval; - if (refreshInterval > 0) { + // FIX #2: the periodic `_slots.rediscover()` is env-tunable AND disable-able via + // REDIS_CLUSTER_REFRESH_INTERVAL (the standing 30s default is unchanged when unset): + // * a LARGER value lengthens the interval; + // * 0 (or any value <= 0) is the DISABLE SENTINEL — no periodic rediscover is + // scheduled at all (event-driven rediscovery on node-error/node-disconnect still + // runs). This lets the infra side run the "lengthen / disable the periodic + // rediscover" experiment (it is the most plausible SEED of the inflight-leak wedge: + // a periodic rediscover racing an in-flight write) as a pure env change, no deploy. + const { enabled: refreshEnabled, intervalMs: refreshInterval } = resolvePeriodicRefresh( + config.clusterRefreshInterval + ); + // Guard against re-entry. setupEnhancedFailover runs on the INITIAL connect() AND is + // re-invoked by forceClusterReconnect (FIX #3) after a self-heal forced reconnect (it's + // passed in as reSetupFailover). Without clearing first, a re-setup would stack a second + // interval + re-wrap close. forceClusterReconnect already clears the interval before its + // destroy(); clearing again here is idempotent and also covers any future re-entry path. + // Net: after any connect()/reconnect there is exactly ONE refresh interval. + clearClusterRefreshInterval(type); + if (refreshEnabled) { const intervalId = setInterval(() => { triggerTopologyRediscovery(baseClient, 'periodic refresh'); }, refreshInterval); @@ -592,21 +854,30 @@ function getBaseClient(type: 'cache' | 'system') { // Store interval for cleanup clusterRefreshIntervals.set(type, intervalId); - // Wrap close to clean up interval - const originalClose = baseClient.close?.bind(baseClient); - if (originalClose) { - (baseClient as any).close = async () => { - const interval = clusterRefreshIntervals.get(type); - if (interval) { - clearInterval(interval); - clusterRefreshIntervals.delete(type); - log(`Cleared topology refresh interval for ${type}`); - } - return originalClose(); - }; + // Wrap close to clean up interval (only wrap once — re-wrapping on each reconnect + // would nest the wrappers). + if (!(baseClient as any).__refreshCloseWrapped) { + const originalClose = baseClient.close?.bind(baseClient); + if (originalClose) { + (baseClient as any).__refreshCloseWrapped = true; + (baseClient as any).close = async () => { + const interval = clusterRefreshIntervals.get(type); + if (interval) { + clearInterval(interval); + clusterRefreshIntervals.delete(type); + log(`Cleared topology refresh interval for ${type}`); + } + return originalClose(); + }; + } } log(`Topology refresh scheduled every ${refreshInterval}ms`); + } else { + // Disable sentinel hit. + log( + `Periodic topology refresh DISABLED (REDIS_CLUSTER_REFRESH_INTERVAL=${refreshInterval} <= 0); event-driven rediscovery still active` + ); } } catch (err) { log(`Enhanced failover setup failed: ${err instanceof Error ? err.message : err}`); @@ -625,6 +896,15 @@ function getBaseClient(type: 'cache' | 'system') { .then(async () => { log(`${type} cluster client connected`); await setupEnhancedFailover(); + // FIX #1: start the inflight-leak self-heal watchdog ONCE per process for the cache + // cluster client. Guarded so a self-heal forced reconnect doesn't stack a second + // watchdog interval. Pass setupEnhancedFailover so forceClusterReconnect re-establishes + // the periodic-rediscover interval after a heal (FIX #3). The forced reconnect does NOT + // re-run this connect() .then callback (it calls baseClient.connect() directly), so the + // re-setup must be threaded through the watchdog, not relied on here. + if (type === 'cache' && !clusterSelfHealInterval) { + startClusterSelfHeal(baseClient, setupEnhancedFailover); + } }) .catch((err) => { log(`Redis connection failed (${type})`, err); @@ -665,6 +945,15 @@ function getClient(type: 'cache' | 'system') { // Decode a packed Buffer, evicting the bad entry if msgpack fails so the next // read falls through to source. Returns null on decode failure, treated as cache miss. + // + // This is the GENERAL decode path for EVERY packed read (get/mGet/sMembers/sPop/ + // hGet/hGetAll/hmGet) across ~30 cache writers. It does plain `unpack(value)` only — + // it deliberately does NOT brotli-decompress, so the 0x01 brotli sentinel is NOT a + // global invariant on all packed values (a bare msgpack `1` packs to <01> and must + // decode as the integer 1, not be mistaken for a compressed payload). Brotli + // decompression is confined to the opt-in compress-aware read path below + // (`get(key, { compress: true })`) — see ./packed-compression for why the + // sentinel is provably collision-free THERE (wrapper objects only). const safeUnpack = (value: Buffer, evict: () => Promise): T | null => { try { return unpack(value) as T; @@ -678,10 +967,42 @@ function getClient(type: 'cache' | 'system') { } }; + // Compress-aware decode — the SYMMETRIC counterpart of `set(..., { compress: true })`, + // used ONLY by the opt-in `get(key, { compress: true })` read path (currently just + // fetchThroughCache's compressed callers, e.g. tensor-metadata full). It awaits the + // async brotli codec and discriminates on the sentinel byte so a legacy uncompressed + // value (written before compression was enabled) still decodes: first byte === sentinel + // → strip + brotli-decompress + unpack; else → unpack as legacy raw. Provably safe HERE + // (and only here) because every value on this path is the `{ data, cachedAt }` wrapper + // object whose msgpack first byte is always a MAP marker (0x80–0x8f / 0xde / 0xdf), + // never 0x01. Preserves safeUnpack's evict-on-failure / fail-open semantics: a + // decompress/unpack throw is treated as a cache miss (null) and evicts the bad entry. + const safeUnpackCompressed = async ( + value: Buffer, + evict: () => Promise + ): Promise => { + try { + return unpack(await decompressPacked(value)) as T; + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + log(`Packed (compressed) unpack failed, evicting bad cache entry: ${msg}`); + evict().catch((e) => + log(`Eviction after unpack failure failed: ${e instanceof Error ? e.message : e}`) + ); + return null; + } + }; + client.packed = { - async get(key: K): Promise { + async get(key: K, packedOptions?: { compress?: boolean }): Promise { const result = await bufferClient.get(key); if (!result) return null; + // Only the opt-in compress-aware read attempts sentinel-detect + brotli-decompress + // (symmetric with `set(..., { compress: true })`). Non-compress callers use the + // general decode path verbatim, so no decompression ever runs for them. + if (packedOptions?.compress) { + return safeUnpackCompressed(result, () => client.unlink(key)); + } return safeUnpack(result, () => client.unlink(key)); }, @@ -693,8 +1014,17 @@ function getClient(type: 'cache' | 'system') { ); }, - async set(key: K, value: T, options?: SetOptions): Promise { - await client.set(key, pack(value), options); + async set( + key: K, + value: T, + options?: SetOptions, + packedOptions?: { compress?: boolean } + ): Promise { + const packed = pack(value); + // Opt-in brotli (sentinel-tagged), async so the codec runs on the libuv threadpool + // and never blocks the event loop. The symmetric read is get(key, { compress: true }). + const payload = packedOptions?.compress ? await compressPacked(packed) : packed; + await client.set(key, payload, options); }, async setNX(key: K, value: T): Promise { @@ -1241,6 +1571,7 @@ export const REDIS_KEYS = { IMAGE_TAGS: 'packed:caches:image-tags', MODEL_VERSION_RESOURCE_INFO: 'packed:caches:model-version-resource-info', TENSOR_METADATA: 'packed:caches:tensor-metadata', + TENSOR_METADATA_SUMMARY: 'packed:caches:tensor-metadata-summary', IMAGE_RESOURCES: 'packed:caches:image-resources', USER_DOWNLOADS: 'packed:caches:user-downloads:v2', MOD_RULES: { diff --git a/packages/civitai-redis/src/cluster-deadline-hits.ts b/packages/civitai-redis/src/cluster-deadline-hits.ts new file mode 100644 index 0000000000..e54a1e335b --- /dev/null +++ b/packages/civitai-redis/src/cluster-deadline-hits.ts @@ -0,0 +1,68 @@ +/** + * In-process sliding-window count of CLUSTER (cache) command-deadline TIMEOUTS + * (the second self-heal trigger signal — see cluster-selfheal.ts). + * + * WHY THIS EXISTS (the bug it fixes): the original self-heal watchdog (FIX #1) triggered + * ONLY on `redis_commands_inflight{client="cluster"}` staying CONTINUOUSLY above a + * threshold for `REDIS_CLUSTER_SELFHEAL_SUSTAINED_MS` (20s). But the per-command deadline + * (`withCommandDeadline`, REDIS_CLUSTER_COMMAND_TIMEOUT_MS = 15s) REJECTS every parked + * command at ~15s and dec's the inflight counter. During a real half-open park the parked + * commands share roughly the same age, so they reject in a batch every ~15s → inflight + * SAWTOOTHS between ~200 and ~0. Each crash below the threshold resets the watchdog's + * sustained-breach timer (cluster-selfheal.ts line "inflight <= threshold → breachStartedAt + * = null"). Because the deadline (15s) is SHORTER than the sustained window (20s), the + * breach timer can never accumulate 20 continuous seconds → self-heal NEVER fires. This was + * confirmed live: 21 pods wedged to inflight≈200 for 6–12 min while + * `civitai_app_redis_selfheal_reconnect_total` stayed 0 across the whole fleet. + * + * THE FIX: trigger self-heal on a signal the deadline-drain CANNOT erase — the RATE of + * deadline timeouts itself. A healthy cluster client NEVER hits the 15s deadline (healthy + * p99 ≈ 23ms); a half-open client hits it constantly (the drains ARE the hits). So "N + * deadline timeouts within the last W ms" is a monotonic, sawtooth-immune "this client is + * wedged" signal. This module is the in-process recorder for that signal: a tiny bounded + * ring of timeout timestamps with a windowed-count read. + * + * Pure (no redis/prom imports) so it is unit-testable in isolation and so the watchdog can + * sample it without a prom dependency — mirroring cluster-inflight.ts. One cluster client per + * process → a module-scoped recorder is correct. + */ + +// Bounded ring of recent deadline-timeout timestamps (ms). Bounded so a sustained wedge +// can't grow this unbounded: once full we overwrite the oldest entry. The window read only +// ever cares about the last RING_CAPACITY hits inside the window, and the trigger threshold +// is far below the capacity, so overwriting older-than-window entries loses nothing useful. +const RING_CAPACITY = 512; +const ring: number[] = []; +let writeIdx = 0; + +/** + * Record one cluster command-deadline timeout. Called by withCommandDeadline's onTimeout + * hook (cluster client only). `now` is injectable for tests; defaults to Date.now. + */ +export function recordClusterDeadlineHit(now: number = Date.now()): void { + if (ring.length < RING_CAPACITY) { + ring.push(now); + } else { + ring[writeIdx] = now; + writeIdx = (writeIdx + 1) % RING_CAPACITY; + } +} + +/** + * Count deadline timeouts recorded within the last `windowMs` (i.e. timestamp > now-windowMs). + * O(RING_CAPACITY) — trivial at a 1s watchdog sample. `now` injectable for tests. + */ +export function countClusterDeadlineHits(windowMs: number, now: number = Date.now()): number { + const cutoff = now - windowMs; + let count = 0; + for (let i = 0; i < ring.length; i++) { + if (ring[i] > cutoff) count++; + } + return count; +} + +/** Reset the ring (used after a self-heal reconnect so the post-heal window starts clean, and in tests). */ +export function resetClusterDeadlineHits(): void { + ring.length = 0; + writeIdx = 0; +} diff --git a/packages/civitai-redis/src/cluster-inflight.ts b/packages/civitai-redis/src/cluster-inflight.ts new file mode 100644 index 0000000000..f7f9982250 --- /dev/null +++ b/packages/civitai-redis/src/cluster-inflight.ts @@ -0,0 +1,60 @@ +/** + * In-process count of in-flight CLUSTER (cache) redis commands (FIX #1/#2 support). + * + * instrumentCommands inc/decs this in lockstep with the + * `redis_commands_inflight{client="cluster"}` gauge — it is the same signal the gauge + * exposes, read locally (the prom-client Gauge doesn't cheaply surface its value) so the + * self-heal watchdog can sample it without a prom dependency. There is one cluster client + * per process, so a module-scoped counter is correct. + * + * EVERY mutation goes through these helpers so the counter has ONE invariant: + * it can never go negative. + * + * WHY the floor matters (FIX #2): a forced self-heal reconnect calls the cluster client's + * `destroy()`, which `flushAll(DisconnectsClientError)` — IMMEDIATELY rejecting every + * in-flight command. Each of those rejected commands then runs its `done()` closure, which + * decrements this counter. The per-closure `dec` guard only prevents a single closure from + * decrementing twice; it does NOT stop N distinct closures from decrementing past 0. Without + * a global floor, a heal that rejected N commands would leave the counter at ≈ −N, so the + * watchdog would then need inflight > threshold + N to ever re-trigger and a SECOND wedge + * would go unhealed. Flooring every decrement at 0 keeps the counter an accurate reflection + * of reality after a heal, so the watchdog can re-arm. (This is why `reset()` is a + * convenience, not a correctness requirement — the floored decrements settle to ~0 on their + * own as the rejected commands drain.) + * + * Pure (no redis/prom imports) so it is unit-testable in isolation, and so the FIX #2 + * reset→decrement interaction can be exercised against the REAL counter logic client.ts uses + * (not a stubbed constant). + */ + +let clusterInflight = 0; + +/** Increment on command start. */ +export function incClusterInflight(): void { + clusterInflight++; +} + +/** + * Decrement on command settle, FLOORED at 0 so a post-heal burst of rejected commands can't + * drive the counter negative (FIX #2). Returns the new value. + */ +export function decClusterInflight(): number { + clusterInflight = Math.max(0, clusterInflight - 1); + return clusterInflight; +} + +/** Read the current count (the value the self-heal watchdog samples). */ +export function getClusterInflight(): number { + return clusterInflight; +} + +/** + * Reset to 0. Called by forceClusterReconnect before it tears the client down. With + * `destroy()` immediately rejecting the in-flight commands, this is belt-and-suspenders: + * the floored decrements from those rejections settle the counter to ~0 regardless. Kept so + * the watchdog's sampled value snaps clean at the trigger instead of decaying over the next + * few ticks. + */ +export function resetClusterInflight(): void { + clusterInflight = 0; +} diff --git a/packages/civitai-redis/src/cluster-selfheal.ts b/packages/civitai-redis/src/cluster-selfheal.ts new file mode 100644 index 0000000000..08a71dc5f9 --- /dev/null +++ b/packages/civitai-redis/src/cluster-selfheal.ts @@ -0,0 +1,305 @@ +/** + * Cluster-client self-heal watchdog (FIX #1 for the node-redis cluster inflight-leak wedge). + * + * WHY (civitai-dp-prod api-primary): a small fraction of cluster command promises get + * ORPHANED across a cluster retry / `_slots.rediscover()` topology refresh and never + * settle. The per-command deadline (withCommandDeadline / REDIS_CLUSTER_COMMAND_TIMEOUT_MS) + * reaps each *individual* orphaned command — it turns the 125s hang into an error so the + * inflight gauge dec's and the handler unparks — but it NEVER resets the wedged + * client/socket. Once a pod starts orphaning, the NEXT command orphans identically, so the + * pod 500s / slow-degrades indefinitely. ONLY a full client reconnect (rebuilds the + * connections + `_slots`) or a process restart clears the orphaned `_execute` promises. + * + * TWO TRIGGERS (either forces a reconnect, subject to the shared cooldown + single-flight): + * + * TRIGGER 1 — DEADLINE-HIT RATE (the real-wave signal, sawtooth-immune): N cluster + * command-deadline TIMEOUTS within a sliding window. A healthy client hits the 15s deadline + * ZERO times; a half-open client hits it on ~every command. This is the trigger that + * actually fires during a real fleet wave (see the bug below). + * + * TRIGGER 2 — SUSTAINED INFLIGHT (legacy continuous-breach): `redis_commands_inflight` + * pinned above a threshold CONTINUOUSLY for the sustained window. Kept as a backstop for + * wedge shapes that leak inflight without deadline-rejecting. + * + * THE BUG TRIGGER 1 FIXES: TRIGGER 2 alone NEVER fired during real waves. The per-command + * deadline (REDIS_CLUSTER_COMMAND_TIMEOUT_MS = 15s) mass-rejects the parked commands every + * ~15s, so inflight SAWTOOTHS to ~0 and the sustained-breach timer (sustainedMs = 20s > 15s) + * resets before it can accumulate 20 continuous seconds. Confirmed live: 21 pods wedged to + * inflight≈200 for 6–12 min with selfheal_reconnect_total = 0 across the fleet. The + * deadline-TIMEOUT rate is immune to that drain (the drains ARE the timeouts). + * + * RECONNECT-STORM SAFETY — three independent brakes: + * 1. SUSTAINED window: a transient spike drops back under the threshold within the window + * and resets the timer → no reconnect. Only a genuine pinned wedge survives the window. + * 2. COOLDOWN: at most ONE reconnect per cooldown, regardless of how long inflight stays + * pinned. After a reconnect the watchdog re-arms only after the cooldown elapses. + * 3. SINGLE-FLIGHT: while a reconnect is in progress, ticks are skipped. + * + * MONEY/ENTITLEMENT SAFETY: a reconnect rejects the pod's in-flight cluster commands. That + * reject surfaces as a NORMAL command error (a rejected promise) on whatever path issued the + * command — it can NEVER silently succeed-or-skip. The cluster client carries cache/metric + * traffic; money/entitlement state lives in Postgres and the sysRedis (single-node) client, + * which this watchdog NEVER touches. Any caller that does issue a money-adjacent cluster + * command sees the same rejection it already gets for any cluster error and must retry/handle + * it — identical to a socketTimeout teardown today. + * + * This module is PURE (no redis/prom imports) so it can be unit-tested by driving `tick()` + * with a fake clock + injected counter/reconnect, mirroring the command-deadline / sentinel + * test pattern. client.ts constructs one instance per cluster client and drives it on an + * interval. + */ + +export interface ClusterSelfHealConfig { + /** Master kill-switch. When false, tick() is a no-op and never reconnects. */ + enabled: boolean; + /** Inflight count strictly above which a pod is considered potentially wedged. */ + inflightThreshold: number; + /** Inflight must stay above the threshold continuously for this long before reconnecting. */ + sustainedMs: number; + /** Minimum wall-clock time between two reconnects. */ + cooldownMs: number; + /** + * DEADLINE-HIT TRIGGER (the sawtooth-immune signal — see deadlineHitWindowMs note below). + * If this many cluster command-deadline TIMEOUTS occur within `deadlineHitWindowMs`, force a + * reconnect IMMEDIATELY (subject to the cooldown), WITHOUT requiring inflight to stay + * continuously above the threshold. <= 0 disables this trigger (inflight path only). + * + * WHY this exists: the inflight-continuity trigger above can NEVER fire during a real + * half-open park, because the per-command deadline (REDIS_CLUSTER_COMMAND_TIMEOUT_MS = 15s) + * mass-rejects the parked commands every ~15s → inflight sawtooths to ~0 → the sustained + * timer (sustainedMs = 20s > 15s) resets before it can accumulate. Confirmed live: 21 pods + * wedged to inflight≈200 for minutes with selfheal_reconnect_total = 0. The deadline-TIMEOUT + * rate is immune to that drain (the drains ARE the timeouts), so a half-open trips this fast. + */ + deadlineHitThreshold: number; + /** + * Sliding window (ms) over which deadlineHitThreshold deadline timeouts are counted. A + * healthy client hits the 15s deadline ZERO times in any window; a half-open client hits it + * continuously. Default sized so a genuine wedge (every cluster read deadline-rejecting) + * trips well inside the kubelet readiness-shed threshold, while a one-off transient slow + * command (a single deadline hit) does not. + */ + deadlineHitWindowMs: number; + /** + * PER-POD RECONNECT JITTER (fleet-stampede brake). Once a tick DECIDES to reconnect, wait a + * random [0, reconnectJitterMs) before actually calling `reconnect()`. This de-correlates a + * SYNCHRONIZED fleet event: if next-redis-cluster itself has a genuine >=15s blip, every pod's + * deadline-hit trigger trips on the same tick — without jitter all ~80-100 pods would + * destroy()+connect() at once, a connection thundering-herd against an already-unhealthy + * cluster. The cooldown + single-flight guards are taken at DECISION time (before the delay), + * so the jitter never queues a second reconnect or shifts the cooldown clock. <= 0 disables + * jitter (reconnect fires immediately — the original behavior). + */ + reconnectJitterMs: number; +} + +export interface ClusterSelfHealDeps { + /** Reads the current in-process cluster inflight count (same source as the gauge). */ + getInflight: () => number; + /** + * Reads the number of cluster command-deadline timeouts within the last `deadlineHitWindowMs` + * (the sawtooth-immune wedge signal). Receives the window so the recorder owns the windowing. + * Optional so existing callers/tests that only drive the inflight path can omit it (treated + * as 0 → deadline trigger inert). + */ + getDeadlineHits?: (windowMs: number) => number; + /** + * Clears the deadline-hit window. Called right after a reconnect is triggered so the + * post-heal window starts clean and the same wedge can't immediately re-trigger inside the + * cooldown. Optional (no-op if omitted). + */ + resetDeadlineHits?: () => void; + /** + * Forces a full client reconnect (destroy → connect, rebuilding `_slots`). The teardown must + * REJECT in-flight commands immediately (client.ts uses the v5 cluster `destroy()`, NOT the + * draining `close()`) so it can't hang and pin `reconnecting` true forever. Rejecting is fine + * — the watchdog logs it and re-arms after the cooldown. Must resolve/settle. + */ + reconnect: () => Promise; + /** Monotonic-ish clock in ms (injected so tests are deterministic). Defaults to Date.now. */ + now?: () => number; + /** + * Resolves after `ms` (injected so the jitter delay is deterministic in tests). Defaults to a + * setTimeout-based sleep. Only ever called with the per-pod jitter delay; never blocks tick(). + */ + delay?: (ms: number) => Promise; + /** + * Returns a float in [0, 1) for the per-pod jitter (injected so tests are deterministic). + * Defaults to Math.random — app runtime, NOT a workflow script, so Math.random is fine here. + */ + random?: () => number; + /** Structured logger. */ + log: (msg: string, ...rest: unknown[]) => void; + /** + * Called once per successful (or attempted) self-heal reconnect with the inflight value at + * trigger time AND which trigger fired ('deadline' = the sawtooth-immune deadline-hit rate; + * 'inflight' = the legacy sustained-inflight breach), so client.ts can increment the + * Prometheus counter (labeled by trigger) + emit a Loki line. Distinguishing the trigger is + * how the next prod wave is confirmed to have fired the DEADLINE path (the one the inflight + * path could never reach) rather than a stray inflight breach. + */ + onReconnect: (inflightAtTrigger: number, trigger: ClusterSelfHealTrigger) => void; +} + +/** Which watchdog trigger forced a given reconnect (for the labeled metric + log line). */ +export type ClusterSelfHealTrigger = 'deadline' | 'inflight'; + +export class ClusterSelfHealWatchdog { + private readonly cfg: ClusterSelfHealConfig; + private readonly deps: Required> & + ClusterSelfHealDeps; + + /** + * Timestamp (ms) at which inflight FIRST crossed above the threshold in the current + * continuous run, or null when inflight is at/under the threshold. Reset whenever inflight + * drops back under the threshold — this is what makes the trigger require a SUSTAINED breach. + */ + private breachStartedAt: number | null = null; + /** Timestamp of the last reconnect (for the cooldown). null = never reconnected. */ + private lastReconnectAt: number | null = null; + /** Single-flight guard: true while a reconnect promise is in flight. */ + private reconnecting = false; + + constructor(cfg: ClusterSelfHealConfig, deps: ClusterSelfHealDeps) { + this.cfg = cfg; + this.deps = { + ...deps, + now: deps.now ?? Date.now, + delay: deps.delay ?? ((ms: number) => new Promise((r) => setTimeout(r, ms))), + random: deps.random ?? Math.random, + }; + } + + /** Expose internal state for assertions/observability (read-only snapshot). */ + getState() { + return { + breachStartedAt: this.breachStartedAt, + lastReconnectAt: this.lastReconnectAt, + reconnecting: this.reconnecting, + }; + } + + /** + * One watchdog sample. Returns true iff this tick TRIGGERED a reconnect. Safe to call on a + * fixed interval. Never throws — a reconnect rejection is caught and logged. + */ + tick(): boolean { + if (!this.cfg.enabled) { + // Kill-switch: also clear any in-progress breach timer so flipping it back on starts clean. + this.breachStartedAt = null; + return false; + } + if (this.reconnecting) return false; // single-flight + + const now = this.deps.now(); + const inflight = this.deps.getInflight(); + + // ── TRIGGER 1: DEADLINE-HIT RATE (sawtooth-immune — the real-wave signal) ────────── + // A half-open park makes ~every cluster command deadline-reject; the per-command deadline + // then drains inflight, which is exactly why the inflight-continuity trigger below never + // fired during real waves. The deadline-TIMEOUT count is immune to that drain. Evaluate it + // FIRST and independently of the inflight breach timer. + const deadlineHits = + this.cfg.deadlineHitThreshold > 0 && this.deps.getDeadlineHits + ? this.deps.getDeadlineHits(this.cfg.deadlineHitWindowMs) + : 0; + const deadlineTriggered = + this.cfg.deadlineHitThreshold > 0 && deadlineHits >= this.cfg.deadlineHitThreshold; + + // ── TRIGGER 2: SUSTAINED INFLIGHT (legacy continuous-breach path) ────────────────── + let inflightTriggered = false; + if (inflight <= this.cfg.inflightThreshold) { + // Healthy / recovered / transient-spike-ended: reset the sustained timer. + this.breachStartedAt = null; + } else if (this.breachStartedAt == null) { + // Inflight just crossed above the threshold. Start the sustained-breach timer. + this.breachStartedAt = now; + } else if (now - this.breachStartedAt >= this.cfg.sustainedMs) { + inflightTriggered = true; + } + + if (!deadlineTriggered && !inflightTriggered) return false; + + // Respect the cooldown — at most one reconnect per cooldown window (either trigger). + if (this.lastReconnectAt != null && now - this.lastReconnectAt < this.cfg.cooldownMs) { + return false; + } + + // ── TRIGGER ────────────────────────────────────────────────────────────────────── + // Mark the reconnect time + clear the breach timer up front so a long-running reconnect + // can't re-trigger and so the cooldown is measured from the trigger, not completion. Also + // clear the deadline-hit window so the post-heal window starts clean (the same wedge can't + // instantly re-count the pre-heal hits). + this.lastReconnectAt = now; + this.breachStartedAt = null; + this.reconnecting = true; + this.deps.resetDeadlineHits?.(); + + this.deps.log( + deadlineTriggered + ? `Cluster self-heal: ${deadlineHits} command-deadline timeouts in ${this.cfg.deadlineHitWindowMs}ms (>= ${this.cfg.deadlineHitThreshold}), inflight=${inflight} — forcing reconnect` + : `Cluster self-heal: inflight pinned at ${inflight} > ${this.cfg.inflightThreshold} for >=${this.cfg.sustainedMs}ms — forcing reconnect` + ); + // onReconnect is a fire-and-forget observability hook (Prom counter + Loki line). A throw + // from it must NOT abort the reconnect or wedge the watchdog (it would leave reconnecting + // pinned true), so it's isolated. + try { + // 'deadline' takes precedence: it's the trigger we expect during a real fleet wave and + // the one we need confirmed at the next wave. (If both conditions are somehow true, the + // deadline-hit rate is the more specific wedge signal.) + this.deps.onReconnect(inflight, deadlineTriggered ? 'deadline' : 'inflight'); + } catch (err) { + this.deps.log( + `Cluster self-heal: onReconnect hook threw (ignored): ${ + err instanceof Error ? err.message : String(err) + }` + ); + } + + // PER-POD JITTER: wait a random [0, reconnectJitterMs) BEFORE the actual reconnect so a + // synchronized fleet event (next-redis-cluster itself blips >=15s → every pod's deadline-hit + // trigger trips on the same tick) smears its destroy()+connect() across the jitter window + // instead of stampeding the cluster on a single tick. The guards above (lastReconnectAt, + // breachStartedAt cleared, reconnecting=true) are ALREADY taken at decision time, so during + // the delay: concurrent ticks single-flight out (reconnecting=true) and the cooldown clock + // is measured from the decision — the delay can't queue a second reconnect or let the window + // re-accumulate into a double-fire. reconnecting stays true until the reconnect itself + // settles, covering the delay + the reconnect. <= 0 jitter skips the delay entirely and + // invokes reconnect() synchronously (the original behavior). + const jitterMs = + this.cfg.reconnectJitterMs > 0 + ? Math.floor(this.deps.random() * this.cfg.reconnectJitterMs) + : 0; + + // The pre-reconnect step: when jitter > 0, park behind the jitter delay; when jitter is + // disabled (0), invoke reconnect() SYNCHRONOUSLY (no delay, no extra microtask) — identical + // to the original behavior. reconnecting stays true across the delay AND the reconnect, so a + // mid-jitter tick single-flights out and the wedge can't double-fire. + const reconnectChain: Promise = + jitterMs > 0 + ? this.deps + .delay(jitterMs) + .then(() => { + this.deps.log(`Cluster self-heal: reconnecting after ${jitterMs}ms jitter`); + return this.deps.reconnect(); + }) + : this.deps.reconnect(); + + // Fire-and-forget; the watchdog stays responsive (single-flight guard prevents overlap). + // Any rejection is logged, not thrown. + void reconnectChain + .then(() => { + this.deps.log('Cluster self-heal: reconnect completed'); + }) + .catch((err: unknown) => { + this.deps.log( + `Cluster self-heal: reconnect failed: ${err instanceof Error ? err.message : String(err)}` + ); + }) + .finally(() => { + this.reconnecting = false; + }); + + return true; + } +} diff --git a/packages/civitai-redis/src/deadline.ts b/packages/civitai-redis/src/deadline.ts index 32cac0d757..1d08e615df 100644 --- a/packages/civitai-redis/src/deadline.ts +++ b/packages/civitai-redis/src/deadline.ts @@ -17,15 +17,32 @@ * `ms <= 0` (or falsy) disables the guard — returns the input promise unchanged, no timer. The * timer is always cleared in finally() (within `ms`), and Promise.race reaps any late rejection * from the orphaned command so it never surfaces as an unhandledRejection. + * + * `onTimeout` (optional) is invoked exactly once iff the deadline fires (the command did NOT + * settle in time). This is the SELF-HEAL TRIGGER SIGNAL: a healthy cluster client never hits the + * deadline, a half-open one hits it constantly, and — unlike the inflight gauge — the + * deadline-drain cannot erase this count (the drains ARE the hits). instrumentCommands wires it to + * recordClusterDeadlineHit; tests inject a spy. It must never throw into the hot path, so it is + * isolated in a try/catch. */ -export function withCommandDeadline(p: Promise, ms: number): Promise { +export function withCommandDeadline( + p: Promise, + ms: number, + onTimeout?: () => void +): Promise { if (!ms || ms <= 0) return p; let timer: ReturnType | undefined; const deadline = new Promise((_, reject) => { - timer = setTimeout( - () => reject(new Error(`redis cluster command timed out after ${ms}ms`)), - ms - ); + timer = setTimeout(() => { + if (onTimeout) { + try { + onTimeout(); + } catch { + // A broken hook (e.g. a throwing recorder) must never break the deadline guard. + } + } + reject(new Error(`redis cluster command timed out after ${ms}ms`)); + }, ms); }); return Promise.race([p, deadline]).finally(() => { if (timer) clearTimeout(timer); diff --git a/packages/civitai-redis/src/env.ts b/packages/civitai-redis/src/env.ts index de781ea32e..15036fbf6a 100644 --- a/packages/civitai-redis/src/env.ts +++ b/packages/civitai-redis/src/env.ts @@ -39,6 +39,31 @@ export const redisEnvSchema = z // minority of cluster `_execute` promises never settle and park the SSR handler ~125s. // Racing against a rejecting deadline guarantees the command settles. 0 = off. REDIS_CLUSTER_COMMAND_TIMEOUT_MS: z.coerce.number().default(15000), + // ── CLUSTER SELF-HEAL WATCHDOG (FIX #1/#2/#3) ────────────────────────────────────── + // Master kill-switch. ON by default (a forced reconnect is the only thing that clears the + // inflight-leak wedge short of a pod restart). Cluster client ONLY — the single-node + // sysRedis client is never reconnected by this watchdog. + REDIS_CLUSTER_SELFHEAL_ENABLED: z.preprocess( + // default true; only the literal string 'false' disables it + (x) => x !== 'false', + z.boolean().default(true) + ), + // Inflight count strictly above which a pod is considered potentially wedged (50 matches + // the binary-wedge observation: healthy ~0, wedged jumps past 50). + REDIS_CLUSTER_SELFHEAL_INFLIGHT_THRESHOLD: z.coerce.number().default(50), + // Inflight must stay ABOVE the threshold continuously for this long before a reconnect. + REDIS_CLUSTER_SELFHEAL_SUSTAINED_MS: z.coerce.number().default(20000), + // Minimum time between two self-heal reconnects (at most one per cooldown). + REDIS_CLUSTER_SELFHEAL_COOLDOWN_MS: z.coerce.number().default(60000), + // How often the watchdog samples inflight (cheap one-gauge read). + REDIS_CLUSTER_SELFHEAL_CHECK_INTERVAL_MS: z.coerce.number().default(1000), + // DEADLINE-HIT TRIGGER (the sawtooth-immune self-heal signal): N cluster command-deadline + // TIMEOUTS within the window force a reconnect. <= 0 disables this trigger. + REDIS_CLUSTER_SELFHEAL_DEADLINE_HIT_THRESHOLD: z.coerce.number().default(10), + REDIS_CLUSTER_SELFHEAL_DEADLINE_HIT_WINDOW_MS: z.coerce.number().default(20000), + // PER-POD RECONNECT JITTER (fleet-stampede brake): wait a random [0, this) before the + // actual reconnect so a synchronized fleet event doesn't stampede the cluster. + REDIS_CLUSTER_SELFHEAL_RECONNECT_JITTER_MS: z.coerce.number().default(1000), // Used only to derive a hostname for the failover feature-flag context NEXTAUTH_URL: z.string().optional(), FLIPT_DEPLOYMENT_ID: z.string().optional(), @@ -79,6 +104,14 @@ function buildEnv() { sysCommandsQueueMaxLength: parsed.data.REDIS_SYS_COMMANDS_QUEUE_MAX_LENGTH, pingIntervalMs: parsed.data.REDIS_PING_INTERVAL_MS, clusterCommandTimeoutMs: parsed.data.REDIS_CLUSTER_COMMAND_TIMEOUT_MS, + clusterSelfHealEnabled: parsed.data.REDIS_CLUSTER_SELFHEAL_ENABLED, + clusterSelfHealInflightThreshold: parsed.data.REDIS_CLUSTER_SELFHEAL_INFLIGHT_THRESHOLD, + clusterSelfHealSustainedMs: parsed.data.REDIS_CLUSTER_SELFHEAL_SUSTAINED_MS, + clusterSelfHealCooldownMs: parsed.data.REDIS_CLUSTER_SELFHEAL_COOLDOWN_MS, + clusterSelfHealCheckIntervalMs: parsed.data.REDIS_CLUSTER_SELFHEAL_CHECK_INTERVAL_MS, + clusterSelfHealDeadlineHitThreshold: parsed.data.REDIS_CLUSTER_SELFHEAL_DEADLINE_HIT_THRESHOLD, + clusterSelfHealDeadlineHitWindowMs: parsed.data.REDIS_CLUSTER_SELFHEAL_DEADLINE_HIT_WINDOW_MS, + clusterSelfHealReconnectJitterMs: parsed.data.REDIS_CLUSTER_SELFHEAL_RECONNECT_JITTER_MS, nextAuthUrl: parsed.data.NEXTAUTH_URL, fliptDeploymentId: parsed.data.FLIPT_DEPLOYMENT_ID, }; diff --git a/packages/civitai-redis/src/packed-compression.ts b/packages/civitai-redis/src/packed-compression.ts new file mode 100644 index 0000000000..515cae8f38 --- /dev/null +++ b/packages/civitai-redis/src/packed-compression.ts @@ -0,0 +1,60 @@ +import zlib from 'zlib'; +import { promisify } from 'util'; + +/** + * Opt-in brotli compression for `redis.packed` values. + * + * A small set of packed caches store large, highly-compressible blobs (e.g. + * tensor-metadata: ~335 KB of repetitive tensor-name strings, measured ~65x with + * brotli quality 6). Compression is OPT-IN per call — most packed values are tiny and + * would only pay overhead. + * + * ASYNC codec: brotli is run via the libuv threadpool (`util.promisify(zlib.brotli*)`) + * rather than the *Sync variants, so a worst-case large checkpoint (~tens of thousands + * of tensors → multi-MB `tensors[]`; the safetensors header read is capped at 64 MiB, + * measured ~36 ms compress / ~5 ms decompress) does NOT block the Node event loop. The + * call sites in redis/client.ts set/get are already async and simply `await` these. + * + * On-disk format for a compressed value is a single SENTINEL prefix byte (0x01 = brotli) + * followed by the brotli stream of the msgpack-packed Buffer. + * + * SENTINEL SCOPE: decompression is CONFINED to the compress-aware read path + * (`redis.packed.get(key, { compress: true })`, used only by `fetchThroughCache` when + * `compress: true`). The general decode path (`safeUnpack`, used by every other packed + * read) NEVER touches this code, so the 0x01 sentinel is NOT a global invariant on all + * packed values — it only applies on the one confined path, where every value is the + * `{ data, cachedAt }` WRAPPER OBJECT (msgpack first byte is always a MAP marker + * 0x80–0x8f / 0xde / 0xdf — never 0x01), making the sentinel provably collision-free. + * Do NOT enable `compress` for a caller that stores a bare scalar (a positive-fixint + * 0x01 would be ambiguous with the sentinel). + */ +export const PACKED_BROTLI_SENTINEL = 0x01; +const PACKED_BROTLI_QUALITY = 6; + +const brotliCompress = promisify(zlib.brotliCompress); +const brotliDecompress = promisify(zlib.brotliDecompress); + +/** Brotli-compress an already-msgpack-packed Buffer and prepend the sentinel byte. */ +export async function compressPacked(packed: Buffer): Promise { + const compressed = await brotliCompress(packed, { + params: { + [zlib.constants.BROTLI_PARAM_QUALITY]: PACKED_BROTLI_QUALITY, + [zlib.constants.BROTLI_PARAM_SIZE_HINT]: packed.length, + }, + }); + return Buffer.concat([Buffer.from([PACKED_BROTLI_SENTINEL]), compressed]); +} + +/** + * Return the raw msgpack Buffer to feed to `unpack()`, transparently handling both the + * brotli-sentinel-prefixed (new) and raw-msgpack (legacy) on-disk formats. + * + * Only the compress-aware read path calls this — see the SENTINEL SCOPE note above for + * why the first-byte sentinel check is collision-free there. + */ +export async function decompressPacked(value: Buffer): Promise { + if (value.length > 0 && value[0] === PACKED_BROTLI_SENTINEL) { + return brotliDecompress(value.subarray(1)); + } + return value; +} diff --git a/packages/civitai-redis/vitest.config.mts b/packages/civitai-redis/vitest.config.mts new file mode 100644 index 0000000000..7eeb3f85df --- /dev/null +++ b/packages/civitai-redis/vitest.config.mts @@ -0,0 +1,8 @@ +import { defineConfig } from 'vitest/config'; + +export default defineConfig({ + test: { + environment: 'node', + include: ['src/**/*.test.ts'], + }, +}); diff --git a/packages/civitai-telemetry/src/client.ts b/packages/civitai-telemetry/src/client.ts index a33cc4afc5..926caa6096 100644 --- a/packages/civitai-telemetry/src/client.ts +++ b/packages/civitai-telemetry/src/client.ts @@ -296,6 +296,27 @@ export const redisCommandDuration = registerHistogram({ buckets: [0.001, 0.005, 0.025, 0.1, 0.5, 1, 2, 5, 10, 30], }); +// Cluster-client SELF-HEAL reconnect counter (FIX #1). Incremented once each time the inflight-leak +// watchdog forces a full cluster reconnect. Healthy pods never touch this; a nonzero rate flags a pod +// that hit the binary-wedge state and was auto-recovered (vs needing a human rolling-restart). `trigger` +// distinguishes the two watchdog paths: 'deadline' = the sawtooth-immune deadline-hit-rate trigger, +// 'inflight' = the legacy sustained-inflight breach. (See @civitai/redis cluster-selfheal.) +export const redisSelfHealReconnectCounter = registerCounterWithLabels({ + name: 'redis_selfheal_reconnect_total', + help: 'Forced cluster-client reconnects by the inflight-leak self-heal watchdog', + labelNames: ['trigger'] as const, +}); + +// Non-critical metric WRITE/LOCK fail-soft counter (FIX #3). Incremented when a metrics:lock setNX/expire +// or an increment hIncrBy hit the short fail-fast timeout (or a redis error) and the call site skipped the +// metric/lock so the user mutation could still succeed. A sustained rate means engagement metrics are +// momentarily under-counting on a wedged pod — never a money/entitlement impact (analytics counters). +export const redisMetricWriteFailSoftCounter = registerCounterWithLabels({ + name: 'redis_metric_write_failsoft_total', + help: 'Non-critical metric write/lock cluster commands that failed soft (timed out/errored, skipped)', + labelNames: ['op'] as const, +}); + // sysRedis Sentinel observability. Uses the `civitai_sysredis_*` metric prefix (NOT civitai_app_*) // to match the dashboard naming, so it needs its own registrar. const SYSREDIS_PREFIX = 'civitai_sysredis_'; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index a42ab7f2ac..6aabf5f1c6 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -35,8 +35,8 @@ importers: specifier: ^0.4.0 version: 0.4.2(@civitai/app-sdk@0.6.0)(react@18.3.1) '@civitai/client': - specifier: 0.2.0-beta.71 - version: 0.2.0-beta.71 + specifier: 0.2.0-beta.73 + version: 0.2.0-beta.73 '@civitai/cybertipline-tools': specifier: ^0.1.0 version: 0.1.0 @@ -571,6 +571,9 @@ importers: superjson: specifier: ^2.2.6 version: 2.2.6 + three: + specifier: ^0.180.0 + version: 0.180.0 trie-memoize: specifier: ^1.2.0 version: 1.2.0 @@ -692,6 +695,9 @@ importers: '@types/sharp': specifier: ^0.31.0 version: 0.31.1 + '@types/three': + specifier: ^0.180.0 + version: 0.180.0 '@types/uuid': specifier: ^9.0.0 version: 9.0.8 @@ -709,13 +715,13 @@ importers: version: 5.62.0(eslint@8.57.1)(typescript@5.9.2) '@vitest/browser': specifier: ^4.0.18 - version: 4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18) + version: 4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18) '@vitest/browser-playwright': specifier: 4.0.18 - version: 4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(playwright@1.57.0)(utf-8-validate@5.0.10)(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18) + version: 4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(playwright@1.57.0)(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18) '@vitest/coverage-v8': specifier: ^4.0.18 - version: 4.0.18(@vitest/browser@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18))(vitest@4.0.18) + version: 4.0.18(@vitest/browser@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18))(vitest@4.0.18) autoprefixer: specifier: ^10.4.19 version: 10.4.21(postcss@8.5.6) @@ -857,10 +863,10 @@ importers: devDependencies: '@sveltejs/adapter-node': specifier: ^5.3.2 - version: 5.5.4(@sveltejs/kit@2.65.1(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))) + version: 5.5.4(@sveltejs/kit@2.66.0(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))) '@sveltejs/kit': specifier: ^2.43.2 - version: 2.65.1(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)) + version: 2.66.0(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)) '@sveltejs/vite-plugin-svelte': specifier: ^6.0.0 version: 6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)) @@ -1434,8 +1440,8 @@ packages: '@civitai/app-sdk': ^0.6.0 react: ^18.0.0 || ^19.0.0 - '@civitai/client@0.2.0-beta.71': - resolution: {integrity: sha512-80FYKe0RiK0a6g41HFujVi8dgi5JOWEJ8ZYh/8e2gq3sUdtHFYeWDVMPcfSXntwLabtLf5LMRT+sDaV+lCjnpA==} + '@civitai/client@0.2.0-beta.73': + resolution: {integrity: sha512-cpnAvXXwnfy++LVNYFrxkzzHZ8wG20e8TtEbdyN+jw/54KO2YTi3r63Rhr+IlbyIM22KIjFIfun8JtDd/R0Beg==} engines: {git: '>=2.11.0', node: '>=18.0.0', npm: '>=7.19.0', yarn: '>=1.7.0'} '@civitai/cybertipline-tools@0.1.0': @@ -1506,6 +1512,9 @@ packages: resolution: {integrity: sha512-QxULHAm7cNu72w97JUNCBFODFaXpbDg+dP8b/oWFAZ2MTRppA3U00Y2L1HqaS4J6yBqxwa/Y3nMBaxVKbB/NsA==} engines: {node: '>=20.19.0'} + '@dimforge/rapier3d-compat@0.12.0': + resolution: {integrity: sha512-uekIGetywIgopfD97oDL5PfeezkFpNhwlzlaEYNOA0N6ghdsOvh/HYjSMek5Q2O1PYvRSDFcqFVJl4r4ZBwOow==} + '@discordjs/builders@1.11.2': resolution: {integrity: sha512-F1WTABdd8/R9D1icJzajC4IuLyyS8f3rTOz66JsSI3pKvpCAtsMBweu8cyNYsIyvcrKAVn9EPK+Psoymq+XC0A==} engines: {node: '>=16.11.0'} @@ -2467,6 +2476,9 @@ packages: '@jridgewell/sourcemap-codec@1.5.5': resolution: {integrity: sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==} + '@jridgewell/trace-mapping@0.3.29': + resolution: {integrity: sha512-uw6guiW/gcAGPDhLmd77/6lW8QLeiV5RUTsAX46Db6oLhGaVj4lhnPwb184s1bkc8kdVg/+h988dro8GRDpmYQ==} + '@jridgewell/trace-mapping@0.3.31': resolution: {integrity: sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw==} @@ -3399,8 +3411,8 @@ packages: cpu: [arm] os: [android] - '@rollup/rollup-android-arm-eabi@4.62.0': - resolution: {integrity: sha512-IPIQ55ythEHkfEd9jMEi32OQ7SxURsGA43JI22lj01OLZNt2NUbJX8YUHxkVWyQ6daHPNn0truF5nSj3DQp6YQ==} + '@rollup/rollup-android-arm-eabi@4.62.2': + resolution: {integrity: sha512-6o7ZLZK+BeenkZCFNDXqpbjw9bD6nuWonvS/lwQJp7NoVVxm6p3qE7qQ5jGuBjiFsgvqjD8mZAU5oWxTmbOeOg==} cpu: [arm] os: [android] @@ -3409,8 +3421,8 @@ packages: cpu: [arm64] os: [android] - '@rollup/rollup-android-arm64@4.62.0': - resolution: {integrity: sha512-M6s9cr10MibETyo8JsOkq+Lo1+lU6hcvb1MApnUql5qte/5hMEgzlN8/ReIKNfRV8rrqX50W1BX9zoUhC192RA==} + '@rollup/rollup-android-arm64@4.62.2': + resolution: {integrity: sha512-BaH7BllCACHoH1LguOU56UItGfUWjujlO65kS9LAodViaN4bwIKd7oeW/ZHJ/4ljr/7MIiENnNy3HJ0zXv8Zkw==} cpu: [arm64] os: [android] @@ -3419,8 +3431,8 @@ packages: cpu: [arm64] os: [darwin] - '@rollup/rollup-darwin-arm64@4.62.0': - resolution: {integrity: sha512-BqCoMoIbn0keKys+dEAdBa70EtOwV1bEsQCUgU9FdiZmmMge/Zk7LlkYGqbrdHR+Frnt0E1FOanly+rlwvvQzw==} + '@rollup/rollup-darwin-arm64@4.62.2': + resolution: {integrity: sha512-v39RCCvj4He82I9sFmk+M1VZ0PLM9sfsLVikjfx2hYBNALhrrOR2D3JjQA6AhlaSOgcR+RzrKY7e1+bT6SUO/A==} cpu: [arm64] os: [darwin] @@ -3429,8 +3441,8 @@ packages: cpu: [x64] os: [darwin] - '@rollup/rollup-darwin-x64@4.62.0': - resolution: {integrity: sha512-SIMzST3VFNXDAbeIWDWiFCNM5qncUBDWaEV7NfE7oZbDt2mgfW4MvbKdbYiGOLoM32gbTv608UMd0XktEYSD7w==} + '@rollup/rollup-darwin-x64@4.62.2': + resolution: {integrity: sha512-yl0y2vq3S3lHeuXhEdss6TWfKW8vkujImO12tn4ZkG/4oghr09LvdYm2RElVjokTQiUvDUGXLGsYeLqUMCKpGA==} cpu: [x64] os: [darwin] @@ -3439,8 +3451,8 @@ packages: cpu: [arm64] os: [freebsd] - '@rollup/rollup-freebsd-arm64@4.62.0': - resolution: {integrity: sha512-ezjfSQMP7ArdUsbBwbQIfwAlhE84I2iVnzQNCFSveqV42q+BmKlzVpf7mxv5EchLcoWU4y6/heFzVg1F+hodUQ==} + '@rollup/rollup-freebsd-arm64@4.62.2': + resolution: {integrity: sha512-tT4pvt4qXD+vEoezupCWi+a1F0vvDiksiHc+PxRlYTOH1I6/X4id9jPxTP+Fg+545euaFT1jJVs4CEdHZAU1vw==} cpu: [arm64] os: [freebsd] @@ -3449,8 +3461,8 @@ packages: cpu: [x64] os: [freebsd] - '@rollup/rollup-freebsd-x64@4.62.0': - resolution: {integrity: sha512-9+qTWGW9AZRhnUgwtTwzNwcPlL87ngkeN0LA+q1bADvmY9aNvWaF2TFW8BZgnQPYxpDI7+rMVLivcd4V737TAQ==} + '@rollup/rollup-freebsd-x64@4.62.2': + resolution: {integrity: sha512-6nU5F2wCW+qvCBhTn1pdIU3bzsIoF7EUwsCDRxilWGprQR6yd508YnH9+OKFCwpfS8pjZqDUmnCAr7exax0XCg==} cpu: [x64] os: [freebsd] @@ -3459,8 +3471,8 @@ packages: cpu: [arm] os: [linux] - '@rollup/rollup-linux-arm-gnueabihf@4.62.0': - resolution: {integrity: sha512-T1dMEQhXA/jkJ/jyMIw9IovK8bSUq7A8kLIlvZTb/6YIVsp2zLavr4F3oyllHWo7eIVJRyE5n3tUjQJEbE1IuQ==} + '@rollup/rollup-linux-arm-gnueabihf@4.62.2': + resolution: {integrity: sha512-n1GJHPOvpIfhi3TmrCeh6S6URt9BFCt0KQE3qvexyGCTAKpR4Lg+eWvNZEqu7epxwus/8ElT3hacYEucm49SZg==} cpu: [arm] os: [linux] @@ -3469,8 +3481,8 @@ packages: cpu: [arm] os: [linux] - '@rollup/rollup-linux-arm-musleabihf@4.62.0': - resolution: {integrity: sha512-2as0LgT7qQpyceQq6VUJYnumUMUrgGQCWIiDIN9DE0/tglsk6o66uCB4f3djRawAltvfCNLyZZrsqbPA6inCsA==} + '@rollup/rollup-linux-arm-musleabihf@4.62.2': + resolution: {integrity: sha512-JqgflS8wEB+UXV/vS1RpRbifGBeN4D5lz8D8oOFbFZw4vedvdOgCFAjfBmIMdW3yL10XpQQ0Ambepw6MXrhOnA==} cpu: [arm] os: [linux] @@ -3479,8 +3491,8 @@ packages: cpu: [arm64] os: [linux] - '@rollup/rollup-linux-arm64-gnu@4.62.0': - resolution: {integrity: sha512-bVURMg+6eNN9C/yc0aVjooZcwTTtYF4YW3xta5pP0//r3o1V8gXEHXWCndj47w/HhwsFroZrFhR+6uQP5T0n0g==} + '@rollup/rollup-linux-arm64-gnu@4.62.2': + resolution: {integrity: sha512-wnFJkogWvN4jm/hQRF2UBaeUmk20j5+DmHvoyWii2b8HJDyvz1MF2OU/6ynXt2KR63rbZLWkFpoytpdc/yBuSA==} cpu: [arm64] os: [linux] @@ -3489,8 +3501,8 @@ packages: cpu: [arm64] os: [linux] - '@rollup/rollup-linux-arm64-musl@4.62.0': - resolution: {integrity: sha512-Ful8pM/2yYI83PViWdFdpZhdI8HJ5qsXANe5atypbHDf+KIBBDsZsbyy8hbXnULVvW9NsTh5DHwbcBftyLTfiw==} + '@rollup/rollup-linux-arm64-musl@4.62.2': + resolution: {integrity: sha512-HVu2bp0zhvJ8xHEV9+UUs7S90VadmBSY3LcIMvozbPo4AuMGDWlz3ymHLHZPX4hR67TKTt8Qp5PJ5RBg/i+RMQ==} cpu: [arm64] os: [linux] @@ -3499,8 +3511,8 @@ packages: cpu: [loong64] os: [linux] - '@rollup/rollup-linux-loong64-gnu@4.62.0': - resolution: {integrity: sha512-9Gp/DgrkzfUBmNPVTyPTvay+4xEP7M/clXpj3efXBcm6uTIVIgDg4rqUpqKXvLEuFRVuEpSAOkhgNeecvaZ4Cg==} + '@rollup/rollup-linux-loong64-gnu@4.62.2': + resolution: {integrity: sha512-mQqqAV8QaoSgr9I2fKDLY2BAVvmKjWoGiu/cSYQonsLvtqwEn1E4QYfnCOcp5zoEqNhsDYin1s6jx/VJmrxlZg==} cpu: [loong64] os: [linux] @@ -3509,8 +3521,8 @@ packages: cpu: [loong64] os: [linux] - '@rollup/rollup-linux-loong64-musl@4.62.0': - resolution: {integrity: sha512-m9tsJz54LUXkSYM8+8PG81B9IKK5r+2T0clMq4QrS16xFosufU7firBDAZEsDheDs7wTlP7h3++S7lMsU955HA==} + '@rollup/rollup-linux-loong64-musl@4.62.2': + resolution: {integrity: sha512-IxKLoxCQ2IWi6bT2akyDUBGsOImDKB+sPp4EsTmwFQ/fMwpCKm8uLSSgP/Kx/QYUgKis6SEZ5/Nlhup0DIA0PQ==} cpu: [loong64] os: [linux] @@ -3519,8 +3531,8 @@ packages: cpu: [ppc64] os: [linux] - '@rollup/rollup-linux-ppc64-gnu@4.62.0': - resolution: {integrity: sha512-3UvJ5PNVU16aJf6M3tFI24pWzAl2/ynfbyRN3ICyQajK1lSkrnVYNnLz3v04J32qKa0FczJc22zeToc0lr2A3w==} + '@rollup/rollup-linux-ppc64-gnu@4.62.2': + resolution: {integrity: sha512-Mk5ha2RQSgyFfmYYLkBpPnUk8D8FriBxesO1u9O75X0mHgXL1UQcH5Itl2lurWL2tj0RxV9b9tJgipac0hRY9A==} cpu: [ppc64] os: [linux] @@ -3529,8 +3541,8 @@ packages: cpu: [ppc64] os: [linux] - '@rollup/rollup-linux-ppc64-musl@4.62.0': - resolution: {integrity: sha512-vRWUAbYLGHBZS6Q8Msb2sfnf1fvJf+47t8l/TwOerM2qArzy+IeNMTHrYLHXh95h8MoatPHI5hhSZNs+mGXKPg==} + '@rollup/rollup-linux-ppc64-musl@4.62.2': + resolution: {integrity: sha512-CjvEnqJL/0/TQ3TXX3OPIJ/kmBellrWd4heXUmHeJlTnmwjKpSJzoehLaL6Xk0ZnMHBu9dZuFADNOrtjF4v+2w==} cpu: [ppc64] os: [linux] @@ -3539,8 +3551,8 @@ packages: cpu: [riscv64] os: [linux] - '@rollup/rollup-linux-riscv64-gnu@4.62.0': - resolution: {integrity: sha512-c00T5SYENHAt86cfW47URaP3Us5vLC/4QO7GYud1G5VNRffCwwCuBspwqYrriuJB+5m0WFzClCn9wed0FBjKvg==} + '@rollup/rollup-linux-riscv64-gnu@4.62.2': + resolution: {integrity: sha512-1SiZbzwdkaDURsew/tSOrooKiYy7EQGT6m8ufavAi9NEyQb/6VuIxFXAL1fqa4iZe3g4NbNk4P7J32z2tw5Mgg==} cpu: [riscv64] os: [linux] @@ -3549,8 +3561,8 @@ packages: cpu: [riscv64] os: [linux] - '@rollup/rollup-linux-riscv64-musl@4.62.0': - resolution: {integrity: sha512-krrCDilhXOwFkSkO3Wm9I/f9H0L92XHHwy2fwxjukxIbh0dem8gZqOW5Y8BsHrpJv5qwlRBV+Wl4ZFyRWhUpwg==} + '@rollup/rollup-linux-riscv64-musl@4.62.2': + resolution: {integrity: sha512-nQts12zJ3NQRoE6uYljOH89v7szzLDvG2JD/vsX+vGXU8w/At1GowTZ5/7qeFQ8m7L55rpR8Okugnuo5bgjy2Q==} cpu: [riscv64] os: [linux] @@ -3559,8 +3571,8 @@ packages: cpu: [s390x] os: [linux] - '@rollup/rollup-linux-s390x-gnu@4.62.0': - resolution: {integrity: sha512-7pfYFSTc4/rUC/FtAI0Qp6QthDBCIi6/AuP1xYqFk5vanI6KnL5dWKP60OM/05LOsbwTmIcvr6eXC4CJuJ75IA==} + '@rollup/rollup-linux-s390x-gnu@4.62.2': + resolution: {integrity: sha512-E9/ll019jhPIJgpzfZoIkBGhcz+kKNgVWYRY0zr9srBdPPFVpvOKW8VaJKUbeK+eZXyQF9ltME+Kk6affeaPgg==} cpu: [s390x] os: [linux] @@ -3569,8 +3581,8 @@ packages: cpu: [x64] os: [linux] - '@rollup/rollup-linux-x64-gnu@4.62.0': - resolution: {integrity: sha512-7SDIalKeIpG0Ifogbbdn58HmSotYMlf23K3dCJEmiVd9Fg36Vmni82iPQec27N3wY4Bvbxftkxz6vSx9OcouTg==} + '@rollup/rollup-linux-x64-gnu@4.62.2': + resolution: {integrity: sha512-5BqxR/pshjey51iliyzTD5Xi3EN0aLmQ2lZ3lvefVV9c82BvrLo2/6OT55iifpWBufs6kdwWbuOKS841DrmK9A==} cpu: [x64] os: [linux] @@ -3579,8 +3591,8 @@ packages: cpu: [x64] os: [linux] - '@rollup/rollup-linux-x64-musl@4.62.0': - resolution: {integrity: sha512-eRZevouTH2i1HeAVLqJuLnt256krQkGY0TN6WsTmsIhuzbh457HuWDMakKwmi0Cjadux983CoSr8Lim2QhUIFw==} + '@rollup/rollup-linux-x64-musl@4.62.2': + resolution: {integrity: sha512-uNN83XxQrRAh/w0/pmAfibcwyb6YWt4gP+dpnQKPVJshAloQ785ii8CT8ZCIxkGg9opVsvAlGhFitSm6D1Jjpg==} cpu: [x64] os: [linux] @@ -3589,8 +3601,8 @@ packages: cpu: [x64] os: [openbsd] - '@rollup/rollup-openbsd-x64@4.62.0': - resolution: {integrity: sha512-3oVS7FLGa4U1qcvao9ylGxrjXZyUQqR8UwxEcnUEyPX53O/C/mKDZegNXTdHCP+h3e6ta/f1EN38Yif1mmZHYg==} + '@rollup/rollup-openbsd-x64@4.62.2': + resolution: {integrity: sha512-srjEIxSH3LRnJN6THczDHWQplqEMFiAJrTab0msUryh9kwNpkICf3Ea6q6MN/2cZwRFUNx5w+h6Hpi4QuHS6Zg==} cpu: [x64] os: [openbsd] @@ -3599,8 +3611,8 @@ packages: cpu: [arm64] os: [openharmony] - '@rollup/rollup-openharmony-arm64@4.62.0': - resolution: {integrity: sha512-yTB9TgfWj5wHe5QgktAgXTLLot1gvEjl1NiPPAUiCs4oPrIWFl5V4nC3GrkNdj9LaAU4s94nVrGbGOCqUpyWsg==} + '@rollup/rollup-openharmony-arm64@4.62.2': + resolution: {integrity: sha512-8hOJnxgbyObnCm5AlRA3A931xX19xq80RjVTKgJOvEKWqJruP/Uf12IbAOaDjjEXYRewwHLfmF0YRIdK3OwKWA==} cpu: [arm64] os: [openharmony] @@ -3609,8 +3621,8 @@ packages: cpu: [arm64] os: [win32] - '@rollup/rollup-win32-arm64-msvc@4.62.0': - resolution: {integrity: sha512-5LOhoaesY3doG1c+ac/2JtgREpKoJr5bUHH8tKY0V8di7+uSV6BwLs2PlR0/yzefGOkR+wE7ZolZphHCsyG5Rw==} + '@rollup/rollup-win32-arm64-msvc@4.62.2': + resolution: {integrity: sha512-mmF4AY1i0hG/bLWUctUq59gtmgaSIRa3cu/A3JFRp/sCNEme2bgDEiDS22P9FbnJB8NJNF4jPJiSP5RHQpUTDg==} cpu: [arm64] os: [win32] @@ -3619,8 +3631,8 @@ packages: cpu: [ia32] os: [win32] - '@rollup/rollup-win32-ia32-msvc@4.62.0': - resolution: {integrity: sha512-yYkWHhmbhRTWTnWos5HC4GcPQfjlzzCNbM9e/+GXrLuaBXYA3qSDR9f0Vgufd5S8yX81U8jPKp7ZnAjZFMtRnw==} + '@rollup/rollup-win32-ia32-msvc@4.62.2': + resolution: {integrity: sha512-DZgkknc6jhHrk46V25vbAM0zZkyP0nSDkJB8/dRkLTxv470dOmWDqGoEJl/9A0dFfS7yE3REOwNDxpHwSLSt0Q==} cpu: [ia32] os: [win32] @@ -3629,8 +3641,8 @@ packages: cpu: [x64] os: [win32] - '@rollup/rollup-win32-x64-gnu@4.62.0': - resolution: {integrity: sha512-SoTb6lPg25xZlA2ibwQ++ahCCnH+FP0qmEuafMJ4gznZKOlXioKEAeJLgCrqjM98ACziXM9V1amFjICVL4IFoA==} + '@rollup/rollup-win32-x64-gnu@4.62.2': + resolution: {integrity: sha512-T6xr6ucWSFto+VGajA8YH26LdpHRuP4YLHEKAtCWvJDOlnmWcDZVCI2Jmjr+IFHDlt2zRaTAKE4tfjTaWLgJBg==} cpu: [x64] os: [win32] @@ -3639,8 +3651,8 @@ packages: cpu: [x64] os: [win32] - '@rollup/rollup-win32-x64-msvc@4.62.0': - resolution: {integrity: sha512-5L+T1fMX4RIEBoZzT0+sQ0PhTS36NULFmMXtl1TZo44TMAROIMHbZufSOjVWt/Y622BtxgxtaNOokbTDvfsrZA==} + '@rollup/rollup-win32-x64-msvc@4.62.2': + resolution: {integrity: sha512-BfzEnDJOt9T8M989/lA37EcJgat01wLRnoi5dQf3QzOH7jzpqTAzdDbVfRljVr5r+jzKqpbHeyOfAaXxAd0PAA==} cpu: [x64] os: [win32] @@ -4001,8 +4013,8 @@ packages: peerDependencies: '@sveltejs/kit': ^2.4.0 - '@sveltejs/kit@2.65.1': - resolution: {integrity: sha512-Sa1rFYYqBB+zv3rIxAg/CsFskR/x4aj5BY/hvLxBd9r/mqbipxM945As1K3PqsDicJAyekPR0BlWoVIiw2OHYg==} + '@sveltejs/kit@2.66.0': + resolution: {integrity: sha512-7nN4Ur4+nofZ36DVo83JbRe02m61Vc+I441mML/DYa1pUTZ/x26+lbrdqPen8gjmsUc6flMtHEqAtn0UfmfvAw==} engines: {node: '>=18.13'} hasBin: true peerDependencies: @@ -4471,6 +4483,9 @@ packages: cpu: [arm64] os: [win32] + '@tweenjs/tween.js@23.1.3': + resolution: {integrity: sha512-vJmvvwFxYuGnF2axRtPYocag6Clbb5YS7kLL+SO/TeVFzHqDIWrNKYtcsPMibjDx9O+bu+psAy9NKfWklassUA==} + '@tybys/wasm-util@0.10.1': resolution: {integrity: sha512-9tTaPJLSiejZKx+Bmog4uSubteqTvFrVrURwkmHixBo0G4seD0zUxp98E1DzUBJxLQ3NPwXrGKDiVjwx/DpPsg==} @@ -4692,12 +4707,18 @@ packages: '@types/sharp@0.31.1': resolution: {integrity: sha512-5nWwamN9ZFHXaYEincMSuza8nNfOof8nmO+mcI+Agx1uMUk4/pQnNIcix+9rLPXzKrm1pS34+6WRDbDV0Jn7ag==} + '@types/stats.js@0.17.4': + resolution: {integrity: sha512-jIBvWWShCvlBqBNIZt0KAshWpvSjhkwkEu4ZUcASoAvhmrgAUI2t1dXrjSL4xXVLB4FznPrIsX3nKXFl/Dt4vA==} + '@types/statuses@2.0.6': resolution: {integrity: sha512-xMAgYwceFhRA2zY+XbEA7mxYbA093wdiW8Vu6gZPGWy9cmOyU9XesH1tNcEWsKFd5Vzrqx5T3D38PWx1FIIXkA==} '@types/stream-to-blob@2.0.0': resolution: {integrity: sha512-tZpPRVwoUMe4ZHr0xt2+RyvkqKFoKzdyCUuzW3NxC4HKxbNEwCMLZiumMuZ+FaizAQJx3rUXVJNXpMCnnOffAw==} + '@types/three@0.180.0': + resolution: {integrity: sha512-ykFtgCqNnY0IPvDro7h+9ZeLY+qjgUWv+qEvUt84grhenO60Hqd4hScHE7VTB9nOQ/3QM8lkbNE+4vKjEpUxKg==} + '@types/tough-cookie@4.0.5': resolution: {integrity: sha512-/Ad8+nIOV7Rl++6f1BdKxFSMgmoqEoYbHRpPcx3JEfv8VRsQe9Z4mCXeJBzxs7mbHY/XOZZuXlRNfhpVPbs6ZA==} @@ -4725,6 +4746,9 @@ packages: '@types/vimeo__player@2.18.3': resolution: {integrity: sha512-IzSzb6doT4I4uAnBHa+mBCiNtK7iAllEJjtpkX0sKY6/s1Vi+aX1134IAiPgiyFlMvFab/oZQpSeccK4r0/T2A==} + '@types/webxr@0.5.24': + resolution: {integrity: sha512-h8fgEd/DpoS9CBrjEQXR+dIDraopAEfu4wYVNY2tEPwk60stPWhvZMf4Foo5FakuQ7HFZoa8WceaWFervK2Ovg==} + '@types/whatwg-mimetype@3.0.2': resolution: {integrity: sha512-c2AKvDT8ToxLIOUlN51gTiHXflsfIFisS4pO7pDPoKouJCESkhZnEy623gwP9laCy5lnLDAw1vAzu2vM2YLOrA==} @@ -5069,6 +5093,9 @@ packages: '@webassemblyjs/wast-printer@1.14.1': resolution: {integrity: sha512-kPSSXE6De1XOR820C90RIo2ogvZG+c3KiHzqUoO/F34Y2shGzesfqv7o57xrxovZJH/MetF5UjroJ/R/3isoiw==} + '@webgpu/types@0.1.70': + resolution: {integrity: sha512-LFiNHHKMvmAEvwVew3JLJmTdShhbdwRFSImUshGhE2mGE8ybQzIo63l5uRp+YKnNx+8Qno8Kf6gN+DKMreIJCA==} + '@xmldom/xmldom@0.9.8': resolution: {integrity: sha512-p96FSY54r+WJ50FIOsCOjyj/wavs8921hG5+kVMmZgKcvIKxMXHTrjNJvRgWa/zuX3B6t2lijLNFaOyuxUH+2A==} engines: {node: '>=14.6'} @@ -6593,8 +6620,8 @@ packages: resolution: {integrity: sha512-ca9pw9fomFcKPvFLXhBKUK90ZvGibiGOvRJNbjljY7s7uq/5YO4BOzcYtJqExdx99rF6aAcnRxHmcUHcz6sQsg==} engines: {node: '>=0.10'} - esrap@2.2.11: - resolution: {integrity: sha512-gPdx+I+BjYEinNMQaBXFjbaJVyoPMU4ZODg5mE+M4DqVG9VusAVHHjcBX+zqyITlI0DIARwDMMzZwAWj36dRoQ==} + esrap@2.2.12: + resolution: {integrity: sha512-On0QbLyaiAkVC4eXtgnXK9Kh2opit+3rcUSOc45DqJ2s/X2eXAHsGOKRSJ6IDagQEW5vPyivANfXUiqgXC67Rw==} peerDependencies: '@typescript-eslint/types': ^8.2.0 peerDependenciesMeta: @@ -6754,6 +6781,14 @@ packages: fbjs@2.0.0: resolution: {integrity: sha512-8XA8ny9ifxrAWlyhAbexXcs3rRMtxWcs3M0lctLfB49jRDHiaxj+Mo0XxbwE7nKZYzgCFoq64FS+WFd4IycPPQ==} + fdir@6.4.6: + resolution: {integrity: sha512-hiFoqpyZcfNm1yc4u8oWCf9A2c4D3QjCrks3zmoVKVxpQRzmPNar1hUJcBG2RQHvEVGDN+Jm81ZheVLAQMK6+w==} + peerDependencies: + picomatch: ^3 || ^4 + peerDependenciesMeta: + picomatch: + optional: true + fdir@6.5.0: resolution: {integrity: sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==} engines: {node: '>=12.0.0'} @@ -6773,6 +6808,9 @@ packages: fetch-cookie@2.2.0: resolution: {integrity: sha512-h9AgfjURuCgA2+2ISl8GbavpUdR+WGAM2McW/ovn4tVccegp8ZqCKWSBR8uRdM8dDNlx5WdKRWxBYUwteLDCNQ==} + fflate@0.8.3: + resolution: {integrity: sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA==} + file-entry-cache@6.0.1: resolution: {integrity: sha512-7Gps/XWymbLk2QLYK4NzpMOrYjMhdIxXuIvy2QBsLE6ljuodKvdkWs/cpyJJ3CVIVpH0Oi1Hvg1ovbMzLdFBBg==} engines: {node: ^10.12.0 || >=12.0.0} @@ -8103,6 +8141,9 @@ packages: resolution: {integrity: sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg==} engines: {node: '>= 8'} + meshoptimizer@0.22.0: + resolution: {integrity: sha512-IebiK79sqIy+E4EgOr+CAw+Ke8hAspXKzBd0JdgEmPHiAwmvEj2S4h1rfvo+o/BnfEYd/jAOg5IeeIjzlzSnDg==} + micromark-core-commonmark@2.0.3: resolution: {integrity: sha512-RDBrHEMSxVFLg6xvnXmb1Ayr2WzLAWjeSATAoxwKYJV94TeNavgoIdA0a9ytzDSVzBy2YKFK+emCPOEibLeCrg==} @@ -9653,8 +9694,8 @@ packages: engines: {node: '>=18.0.0', npm: '>=8.0.0'} hasBin: true - rollup@4.62.0: - resolution: {integrity: sha512-nc72Wgq62I7rtDV4izT5/aaS0zxy3kttkinf9586ApknY3jZO9NYsmtc24fUckA0X7Q2v+ML4a15pdUlV5V/jA==} + rollup@4.62.2: + resolution: {integrity: sha512-RFnrW4lhXA3s3eqHDZvN654g8OTjzRfqpIRJYczCGB6HzphckVAi/Qh4tbPUbRuDi7s1Llv8g/NspLkttY3gTA==} engines: {node: '>=18.0.0', npm: '>=8.0.0'} hasBin: true @@ -10286,6 +10327,9 @@ packages: third-party-capital@1.0.20: resolution: {integrity: sha512-oB7yIimd8SuGptespDAZnNkzIz+NWaJCu2RMsbs4Wmp9zSDUM8Nhi3s2OOcqYuv3mN4hitXc8DVx+LyUmbUDiA==} + three@0.180.0: + resolution: {integrity: sha512-o+qycAMZrh+TsE01GqWUxUIKR1AL0S8pq7zDkYOQw8GqfX8b8VoCKYUoHbhiX5j+7hr8XsuHDVU6+gkQJQKg9w==} + through2@4.0.2: resolution: {integrity: sha512-iOqSav00cVxEEICeD7TjLB1sueEL+81Wpzp2bY17uZjZN0pWZPuo4suZ/61VujxmqSGFfgOcNuTZ85QJwNZQpw==} @@ -10305,6 +10349,9 @@ packages: tinybench@2.9.0: resolution: {integrity: sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==} + tinyexec@1.0.1: + resolution: {integrity: sha512-5uC6DDlmeqiOwCPmK9jMSdOuZTh8bU39Ys6yidB+UTt5hfZUPGAypSgFRiEp+jbi9qH40BLDvy85jIU88wKSqw==} + tinyexec@1.0.2: resolution: {integrity: sha512-W/KYk+NFhkmsYpuHq5JykngiOCnxeVL8v8dFnqxSD8qEEdRfXk1SDM6JzNqcERbcGYj9tMrDQBYV9cjgnunFIg==} engines: {node: '>=18'} @@ -11910,7 +11957,7 @@ snapshots: '@babel/parser': 7.28.0 '@babel/types': 7.28.2 '@jridgewell/gen-mapping': 0.3.12 - '@jridgewell/trace-mapping': 0.3.31 + '@jridgewell/trace-mapping': 0.3.29 jsesc: 3.1.0 '@babel/generator@7.29.1': @@ -12066,7 +12113,7 @@ snapshots: '@civitai/app-sdk': 0.6.0 react: 18.3.1 - '@civitai/client@0.2.0-beta.71': + '@civitai/client@0.2.0-beta.73': dependencies: '@hey-api/client-fetch': 0.1.14 rfc6902: 5.2.0 @@ -12148,6 +12195,8 @@ snapshots: '@csstools/css-tokenizer@4.0.0': optional: true + '@dimforge/rapier3d-compat@0.12.0': {} + '@discordjs/builders@1.11.2': dependencies: '@discordjs/formatters': 0.6.1 @@ -12890,6 +12939,11 @@ snapshots: '@jridgewell/sourcemap-codec@1.5.5': {} + '@jridgewell/trace-mapping@0.3.29': + dependencies: + '@jridgewell/resolve-uri': 3.1.2 + '@jridgewell/sourcemap-codec': 1.5.5 + '@jridgewell/trace-mapping@0.3.31': dependencies: '@jridgewell/resolve-uri': 3.1.2 @@ -13049,11 +13103,11 @@ snapshots: '@mdx-js/mdx@3.1.1': dependencies: - '@types/estree': 1.0.9 + '@types/estree': 1.0.8 '@types/estree-jsx': 1.0.5 '@types/hast': 3.0.4 '@types/mdx': 2.0.13 - acorn: 8.16.0 + acorn: 8.15.0 collapse-white-space: 2.1.0 devlop: 1.1.0 estree-util-is-identifier-name: 3.0.0 @@ -13062,7 +13116,7 @@ snapshots: hast-util-to-jsx-runtime: 2.3.6 markdown-extensions: 2.0.0 recma-build-jsx: 1.0.0 - recma-jsx: 1.0.1(acorn@8.16.0) + recma-jsx: 1.0.1(acorn@8.15.0) recma-stringify: 1.0.0 rehype-recma: 1.0.0 remark-mdx: 3.1.1 @@ -14023,9 +14077,9 @@ snapshots: '@rolldown/pluginutils@1.0.0-beta.27': {} - '@rollup/plugin-commonjs@29.0.3(rollup@4.62.0)': + '@rollup/plugin-commonjs@29.0.3(rollup@4.62.2)': dependencies: - '@rollup/pluginutils': 5.4.0(rollup@4.62.0) + '@rollup/pluginutils': 5.4.0(rollup@4.62.2) commondir: 1.0.1 estree-walker: 2.0.2 fdir: 6.5.0(picomatch@4.0.3) @@ -14033,185 +14087,185 @@ snapshots: magic-string: 0.30.21 picomatch: 4.0.3 optionalDependencies: - rollup: 4.62.0 + rollup: 4.62.2 - '@rollup/plugin-json@6.1.0(rollup@4.62.0)': + '@rollup/plugin-json@6.1.0(rollup@4.62.2)': dependencies: - '@rollup/pluginutils': 5.4.0(rollup@4.62.0) + '@rollup/pluginutils': 5.4.0(rollup@4.62.2) optionalDependencies: - rollup: 4.62.0 + rollup: 4.62.2 - '@rollup/plugin-node-resolve@16.0.3(rollup@4.62.0)': + '@rollup/plugin-node-resolve@16.0.3(rollup@4.62.2)': dependencies: - '@rollup/pluginutils': 5.4.0(rollup@4.62.0) + '@rollup/pluginutils': 5.4.0(rollup@4.62.2) '@types/resolve': 1.20.2 deepmerge: 4.3.1 is-module: 1.0.0 resolve: 1.22.10 optionalDependencies: - rollup: 4.62.0 + rollup: 4.62.2 '@rollup/pluginutils@4.2.1': dependencies: estree-walker: 2.0.2 picomatch: 2.3.1 - '@rollup/pluginutils@5.4.0(rollup@4.62.0)': + '@rollup/pluginutils@5.4.0(rollup@4.62.2)': dependencies: '@types/estree': 1.0.9 estree-walker: 2.0.2 picomatch: 4.0.3 optionalDependencies: - rollup: 4.62.0 + rollup: 4.62.2 '@rollup/rollup-android-arm-eabi@4.56.0': optional: true - '@rollup/rollup-android-arm-eabi@4.62.0': + '@rollup/rollup-android-arm-eabi@4.62.2': optional: true '@rollup/rollup-android-arm64@4.56.0': optional: true - '@rollup/rollup-android-arm64@4.62.0': + '@rollup/rollup-android-arm64@4.62.2': optional: true '@rollup/rollup-darwin-arm64@4.56.0': optional: true - '@rollup/rollup-darwin-arm64@4.62.0': + '@rollup/rollup-darwin-arm64@4.62.2': optional: true '@rollup/rollup-darwin-x64@4.56.0': optional: true - '@rollup/rollup-darwin-x64@4.62.0': + '@rollup/rollup-darwin-x64@4.62.2': optional: true '@rollup/rollup-freebsd-arm64@4.56.0': optional: true - '@rollup/rollup-freebsd-arm64@4.62.0': + '@rollup/rollup-freebsd-arm64@4.62.2': optional: true '@rollup/rollup-freebsd-x64@4.56.0': optional: true - '@rollup/rollup-freebsd-x64@4.62.0': + '@rollup/rollup-freebsd-x64@4.62.2': optional: true '@rollup/rollup-linux-arm-gnueabihf@4.56.0': optional: true - '@rollup/rollup-linux-arm-gnueabihf@4.62.0': + '@rollup/rollup-linux-arm-gnueabihf@4.62.2': optional: true '@rollup/rollup-linux-arm-musleabihf@4.56.0': optional: true - '@rollup/rollup-linux-arm-musleabihf@4.62.0': + '@rollup/rollup-linux-arm-musleabihf@4.62.2': optional: true '@rollup/rollup-linux-arm64-gnu@4.56.0': optional: true - '@rollup/rollup-linux-arm64-gnu@4.62.0': + '@rollup/rollup-linux-arm64-gnu@4.62.2': optional: true '@rollup/rollup-linux-arm64-musl@4.56.0': optional: true - '@rollup/rollup-linux-arm64-musl@4.62.0': + '@rollup/rollup-linux-arm64-musl@4.62.2': optional: true '@rollup/rollup-linux-loong64-gnu@4.56.0': optional: true - '@rollup/rollup-linux-loong64-gnu@4.62.0': + '@rollup/rollup-linux-loong64-gnu@4.62.2': optional: true '@rollup/rollup-linux-loong64-musl@4.56.0': optional: true - '@rollup/rollup-linux-loong64-musl@4.62.0': + '@rollup/rollup-linux-loong64-musl@4.62.2': optional: true '@rollup/rollup-linux-ppc64-gnu@4.56.0': optional: true - '@rollup/rollup-linux-ppc64-gnu@4.62.0': + '@rollup/rollup-linux-ppc64-gnu@4.62.2': optional: true '@rollup/rollup-linux-ppc64-musl@4.56.0': optional: true - '@rollup/rollup-linux-ppc64-musl@4.62.0': + '@rollup/rollup-linux-ppc64-musl@4.62.2': optional: true '@rollup/rollup-linux-riscv64-gnu@4.56.0': optional: true - '@rollup/rollup-linux-riscv64-gnu@4.62.0': + '@rollup/rollup-linux-riscv64-gnu@4.62.2': optional: true '@rollup/rollup-linux-riscv64-musl@4.56.0': optional: true - '@rollup/rollup-linux-riscv64-musl@4.62.0': + '@rollup/rollup-linux-riscv64-musl@4.62.2': optional: true '@rollup/rollup-linux-s390x-gnu@4.56.0': optional: true - '@rollup/rollup-linux-s390x-gnu@4.62.0': + '@rollup/rollup-linux-s390x-gnu@4.62.2': optional: true '@rollup/rollup-linux-x64-gnu@4.56.0': optional: true - '@rollup/rollup-linux-x64-gnu@4.62.0': + '@rollup/rollup-linux-x64-gnu@4.62.2': optional: true '@rollup/rollup-linux-x64-musl@4.56.0': optional: true - '@rollup/rollup-linux-x64-musl@4.62.0': + '@rollup/rollup-linux-x64-musl@4.62.2': optional: true '@rollup/rollup-openbsd-x64@4.56.0': optional: true - '@rollup/rollup-openbsd-x64@4.62.0': + '@rollup/rollup-openbsd-x64@4.62.2': optional: true '@rollup/rollup-openharmony-arm64@4.56.0': optional: true - '@rollup/rollup-openharmony-arm64@4.62.0': + '@rollup/rollup-openharmony-arm64@4.62.2': optional: true '@rollup/rollup-win32-arm64-msvc@4.56.0': optional: true - '@rollup/rollup-win32-arm64-msvc@4.62.0': + '@rollup/rollup-win32-arm64-msvc@4.62.2': optional: true '@rollup/rollup-win32-ia32-msvc@4.56.0': optional: true - '@rollup/rollup-win32-ia32-msvc@4.62.0': + '@rollup/rollup-win32-ia32-msvc@4.62.2': optional: true '@rollup/rollup-win32-x64-gnu@4.56.0': optional: true - '@rollup/rollup-win32-x64-gnu@4.62.0': + '@rollup/rollup-win32-x64-gnu@4.62.2': optional: true '@rollup/rollup-win32-x64-msvc@4.56.0': optional: true - '@rollup/rollup-win32-x64-msvc@4.62.0': + '@rollup/rollup-win32-x64-msvc@4.62.2': optional: true '@rtsao/scc@1.1.0': {} @@ -14741,15 +14795,15 @@ snapshots: dependencies: acorn: 8.16.0 - '@sveltejs/adapter-node@5.5.4(@sveltejs/kit@2.65.1(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))': + '@sveltejs/adapter-node@5.5.4(@sveltejs/kit@2.66.0(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))': dependencies: - '@rollup/plugin-commonjs': 29.0.3(rollup@4.62.0) - '@rollup/plugin-json': 6.1.0(rollup@4.62.0) - '@rollup/plugin-node-resolve': 16.0.3(rollup@4.62.0) - '@sveltejs/kit': 2.65.1(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)) - rollup: 4.62.0 + '@rollup/plugin-commonjs': 29.0.3(rollup@4.62.2) + '@rollup/plugin-json': 6.1.0(rollup@4.62.2) + '@rollup/plugin-node-resolve': 16.0.3(rollup@4.62.2) + '@sveltejs/kit': 2.66.0(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)) + rollup: 4.62.2 - '@sveltejs/kit@2.65.1(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))': + '@sveltejs/kit@2.66.0(@opentelemetry/api@1.9.0)(@sveltejs/vite-plugin-svelte@6.2.4(svelte@5.56.3(@typescript-eslint/types@8.60.1))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)))(svelte@5.56.3(@typescript-eslint/types@8.60.1))(typescript@5.9.2)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))': dependencies: '@standard-schema/spec': 1.0.0 '@sveltejs/acorn-typescript': 1.0.10(acorn@8.16.0) @@ -15185,6 +15239,8 @@ snapshots: '@turbo/windows-arm64@2.9.18': optional: true + '@tweenjs/tween.js@23.1.3': {} + '@tybys/wasm-util@0.10.1': dependencies: tslib: 2.8.1 @@ -15414,12 +15470,24 @@ snapshots: dependencies: '@types/node': 20.19.9 + '@types/stats.js@0.17.4': {} + '@types/statuses@2.0.6': {} '@types/stream-to-blob@2.0.0': dependencies: '@types/node': 20.19.9 + '@types/three@0.180.0': + dependencies: + '@dimforge/rapier3d-compat': 0.12.0 + '@tweenjs/tween.js': 23.1.3 + '@types/stats.js': 0.17.4 + '@types/webxr': 0.5.24 + '@webgpu/types': 0.1.70 + fflate: 0.8.3 + meshoptimizer: 0.22.0 + '@types/tough-cookie@4.0.5': {} '@types/triple-beam@1.3.5': {} @@ -15438,6 +15506,8 @@ snapshots: '@types/vimeo__player@2.18.3': {} + '@types/webxr@0.5.24': {} + '@types/whatwg-mimetype@3.0.2': {} '@types/ws@7.4.7': @@ -15721,19 +15791,6 @@ snapshots: transitivePeerDependencies: - supports-color - '@vitest/browser-playwright@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(playwright@1.57.0)(utf-8-validate@5.0.10)(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18)': - dependencies: - '@vitest/browser': 4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18) - '@vitest/mocker': 4.0.18(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)) - playwright: 1.57.0 - tinyrainbow: 3.1.0 - vitest: 4.0.18(@opentelemetry/api@1.9.0)(@types/node@20.19.9)(@vitest/browser-playwright@4.0.18)(happy-dom@20.9.0(bufferutil@4.0.9)(utf-8-validate@5.0.10))(jiti@2.5.1)(jsdom@27.4.0(@noble/hashes@1.8.0)(bufferutil@4.0.9)(utf-8-validate@5.0.10))(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1) - transitivePeerDependencies: - - bufferutil - - msw - - utf-8-validate - - vite - '@vitest/browser-playwright@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(playwright@1.57.0)(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18)': dependencies: '@vitest/browser': 4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18) @@ -15746,7 +15803,6 @@ snapshots: - msw - utf-8-validate - vite - optional: true '@vitest/browser-playwright@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.3))(playwright@1.57.0)(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18)': dependencies: @@ -15762,23 +15818,6 @@ snapshots: - vite optional: true - '@vitest/browser@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18)': - dependencies: - '@vitest/mocker': 4.0.18(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)) - '@vitest/utils': 4.0.18 - magic-string: 0.30.21 - pixelmatch: 7.1.0 - pngjs: 7.0.0 - sirv: 3.0.2 - tinyrainbow: 3.1.0 - vitest: 4.0.18(@opentelemetry/api@1.9.0)(@types/node@20.19.9)(@vitest/browser-playwright@4.0.18)(happy-dom@20.9.0(bufferutil@4.0.9)(utf-8-validate@5.0.10))(jiti@2.5.1)(jsdom@27.4.0(@noble/hashes@1.8.0)(bufferutil@4.0.9)(utf-8-validate@5.0.10))(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1) - ws: 8.21.0(bufferutil@4.0.9)(utf-8-validate@5.0.10) - transitivePeerDependencies: - - bufferutil - - msw - - utf-8-validate - - vite - '@vitest/browser@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18)': dependencies: '@vitest/mocker': 4.0.18(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1)) @@ -15795,7 +15834,6 @@ snapshots: - msw - utf-8-validate - vite - optional: true '@vitest/browser@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.3))(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18)': dependencies: @@ -15815,7 +15853,7 @@ snapshots: - vite optional: true - '@vitest/coverage-v8@4.0.18(@vitest/browser@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18))(vitest@4.0.18)': + '@vitest/coverage-v8@4.0.18(@vitest/browser@4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18))(vitest@4.0.18)': dependencies: '@bcoe/v8-coverage': 1.0.2 '@vitest/utils': 4.0.18 @@ -15829,7 +15867,7 @@ snapshots: tinyrainbow: 3.0.3 vitest: 4.0.18(@opentelemetry/api@1.9.0)(@types/node@20.19.9)(@vitest/browser-playwright@4.0.18)(happy-dom@20.9.0(bufferutil@4.0.9)(utf-8-validate@5.0.10))(jiti@2.5.1)(jsdom@27.4.0(@noble/hashes@1.8.0)(bufferutil@4.0.9)(utf-8-validate@5.0.10))(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1) optionalDependencies: - '@vitest/browser': 4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18) + '@vitest/browser': 4.0.18(bufferutil@4.0.9)(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(utf-8-validate@5.0.10)(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))(vitest@4.0.18) '@vitest/expect@4.0.18': dependencies: @@ -15840,15 +15878,6 @@ snapshots: chai: 6.2.2 tinyrainbow: 3.1.0 - '@vitest/mocker@4.0.18(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(vite@6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))': - dependencies: - '@vitest/spy': 4.0.18 - estree-walker: 3.0.3 - magic-string: 0.30.21 - optionalDependencies: - msw: 2.12.10(@types/node@20.19.9)(typescript@5.9.2) - vite: 6.4.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1) - '@vitest/mocker@4.0.18(msw@2.12.10(@types/node@20.19.9)(typescript@5.9.2))(vite@7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1))': dependencies: '@vitest/spy': 4.0.18 @@ -15967,6 +15996,8 @@ snapshots: '@webassemblyjs/ast': 1.14.1 '@xtuc/long': 4.2.2 + '@webgpu/types@0.1.70': {} + '@xmldom/xmldom@0.9.8': optional: true @@ -16010,6 +16041,10 @@ snapshots: dependencies: acorn: 8.16.0 + acorn-jsx@5.3.2(acorn@8.15.0): + dependencies: + acorn: 8.15.0 + acorn-jsx@5.3.2(acorn@8.16.0): dependencies: acorn: 8.16.0 @@ -17764,7 +17799,7 @@ snapshots: dependencies: estraverse: 5.3.0 - esrap@2.2.11(@typescript-eslint/types@8.60.1): + esrap@2.2.12(@typescript-eslint/types@8.60.1): dependencies: '@jridgewell/sourcemap-codec': 1.5.5 optionalDependencies: @@ -17935,6 +17970,10 @@ snapshots: transitivePeerDependencies: - encoding + fdir@6.4.6(picomatch@4.0.3): + optionalDependencies: + picomatch: 4.0.3 + fdir@6.5.0(picomatch@4.0.3): optionalDependencies: picomatch: 4.0.3 @@ -17951,6 +17990,8 @@ snapshots: set-cookie-parser: 2.7.1 tough-cookie: 4.1.4 + fflate@0.8.3: {} + file-entry-cache@6.0.1: dependencies: flat-cache: 3.2.0 @@ -19639,6 +19680,8 @@ snapshots: merge2@1.4.1: {} + meshoptimizer@0.22.0: {} + micromark-core-commonmark@2.0.3: dependencies: decode-named-character-reference: 1.2.0 @@ -20176,7 +20219,7 @@ snapshots: consola: 3.4.2 pathe: 2.0.3 pkg-types: 2.2.0 - tinyexec: 1.0.2 + tinyexec: 1.0.1 object-assign@4.1.1: {} @@ -20606,7 +20649,7 @@ snapshots: postcss-js: 4.0.1(postcss@8.5.6) postcss-simple-vars: 7.0.1(postcss@8.5.6) sugarss: 5.0.1(postcss@8.5.6) - tinyglobby: 0.2.15 + tinyglobby: 0.2.14 postcss-modules-extract-imports@3.1.0(postcss@8.5.6): dependencies: @@ -21357,10 +21400,10 @@ snapshots: estree-util-build-jsx: 3.0.1 vfile: 6.0.3 - recma-jsx@1.0.1(acorn@8.16.0): + recma-jsx@1.0.1(acorn@8.15.0): dependencies: - acorn: 8.16.0 - acorn-jsx: 5.3.2(acorn@8.16.0) + acorn: 8.15.0 + acorn-jsx: 5.3.2(acorn@8.15.0) estree-util-to-js: 2.0.0 recma-parse: 1.0.0 recma-stringify: 1.0.0 @@ -21582,35 +21625,35 @@ snapshots: '@rollup/rollup-win32-x64-msvc': 4.56.0 fsevents: 2.3.3 - rollup@4.62.0: + rollup@4.62.2: dependencies: '@types/estree': 1.0.9 optionalDependencies: - '@rollup/rollup-android-arm-eabi': 4.62.0 - '@rollup/rollup-android-arm64': 4.62.0 - '@rollup/rollup-darwin-arm64': 4.62.0 - '@rollup/rollup-darwin-x64': 4.62.0 - '@rollup/rollup-freebsd-arm64': 4.62.0 - '@rollup/rollup-freebsd-x64': 4.62.0 - '@rollup/rollup-linux-arm-gnueabihf': 4.62.0 - '@rollup/rollup-linux-arm-musleabihf': 4.62.0 - '@rollup/rollup-linux-arm64-gnu': 4.62.0 - '@rollup/rollup-linux-arm64-musl': 4.62.0 - '@rollup/rollup-linux-loong64-gnu': 4.62.0 - '@rollup/rollup-linux-loong64-musl': 4.62.0 - '@rollup/rollup-linux-ppc64-gnu': 4.62.0 - '@rollup/rollup-linux-ppc64-musl': 4.62.0 - '@rollup/rollup-linux-riscv64-gnu': 4.62.0 - '@rollup/rollup-linux-riscv64-musl': 4.62.0 - '@rollup/rollup-linux-s390x-gnu': 4.62.0 - '@rollup/rollup-linux-x64-gnu': 4.62.0 - '@rollup/rollup-linux-x64-musl': 4.62.0 - '@rollup/rollup-openbsd-x64': 4.62.0 - '@rollup/rollup-openharmony-arm64': 4.62.0 - '@rollup/rollup-win32-arm64-msvc': 4.62.0 - '@rollup/rollup-win32-ia32-msvc': 4.62.0 - '@rollup/rollup-win32-x64-gnu': 4.62.0 - '@rollup/rollup-win32-x64-msvc': 4.62.0 + '@rollup/rollup-android-arm-eabi': 4.62.2 + '@rollup/rollup-android-arm64': 4.62.2 + '@rollup/rollup-darwin-arm64': 4.62.2 + '@rollup/rollup-darwin-x64': 4.62.2 + '@rollup/rollup-freebsd-arm64': 4.62.2 + '@rollup/rollup-freebsd-x64': 4.62.2 + '@rollup/rollup-linux-arm-gnueabihf': 4.62.2 + '@rollup/rollup-linux-arm-musleabihf': 4.62.2 + '@rollup/rollup-linux-arm64-gnu': 4.62.2 + '@rollup/rollup-linux-arm64-musl': 4.62.2 + '@rollup/rollup-linux-loong64-gnu': 4.62.2 + '@rollup/rollup-linux-loong64-musl': 4.62.2 + '@rollup/rollup-linux-ppc64-gnu': 4.62.2 + '@rollup/rollup-linux-ppc64-musl': 4.62.2 + '@rollup/rollup-linux-riscv64-gnu': 4.62.2 + '@rollup/rollup-linux-riscv64-musl': 4.62.2 + '@rollup/rollup-linux-s390x-gnu': 4.62.2 + '@rollup/rollup-linux-x64-gnu': 4.62.2 + '@rollup/rollup-linux-x64-musl': 4.62.2 + '@rollup/rollup-openbsd-x64': 4.62.2 + '@rollup/rollup-openharmony-arm64': 4.62.2 + '@rollup/rollup-win32-arm64-msvc': 4.62.2 + '@rollup/rollup-win32-ia32-msvc': 4.62.2 + '@rollup/rollup-win32-x64-gnu': 4.62.2 + '@rollup/rollup-win32-x64-msvc': 4.62.2 fsevents: 2.3.3 rope-sequence@1.3.4: {} @@ -22233,7 +22276,7 @@ snapshots: clsx: 2.1.1 devalue: 5.8.1 esm-env: 1.2.2 - esrap: 2.2.11(@typescript-eslint/types@8.60.1) + esrap: 2.2.12(@typescript-eslint/types@8.60.1) is-reference: 3.0.3 locate-character: 3.0.0 magic-string: 0.30.21 @@ -22373,6 +22416,8 @@ snapshots: third-party-capital@1.0.20: {} + three@0.180.0: {} + through2@4.0.2: dependencies: readable-stream: 3.6.2 @@ -22387,11 +22432,13 @@ snapshots: tinybench@2.9.0: {} + tinyexec@1.0.1: {} + tinyexec@1.0.2: {} tinyglobby@0.2.14: dependencies: - fdir: 6.5.0(picomatch@4.0.3) + fdir: 6.4.6(picomatch@4.0.3) picomatch: 4.0.3 tinyglobby@0.2.15: @@ -22996,7 +23043,7 @@ snapshots: tinybench: 2.9.0 tinyexec: 1.0.2 tinyglobby: 0.2.15 - tinyrainbow: 3.1.0 + tinyrainbow: 3.0.3 vite: 7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1) why-is-node-running: 2.3.0 optionalDependencies: @@ -23037,7 +23084,7 @@ snapshots: tinybench: 2.9.0 tinyexec: 1.0.2 tinyglobby: 0.2.15 - tinyrainbow: 3.1.0 + tinyrainbow: 3.0.3 vite: 7.3.1(@types/node@20.19.9)(jiti@2.5.1)(sass@1.90.0)(sugarss@5.0.1(postcss@8.5.6))(terser@5.48.0)(tsx@4.20.3)(yaml@2.8.1) why-is-node-running: 2.3.0 optionalDependencies: diff --git a/prisma/migrations/20260526120000_add_3d_models/migration.sql b/prisma/migrations/20260526120000_add_3d_models/migration.sql new file mode 100644 index 0000000000..937b63d2e1 --- /dev/null +++ b/prisma/migrations/20260526120000_add_3d_models/migration.sql @@ -0,0 +1,411 @@ +-- ============================================================ +-- 3D Models (PolyGen / generated content) support — Phase 1 schema +-- Apply manually per project convention (no `prisma migrate deploy`). +-- See docs/3d-models-plan.md for the design. +-- +-- Scope: generated 3D models (Meshy via orchestrator PolyGen). Data +-- structure is also upload-ready (nullable workflowId/sourceImageId, +-- multi-format file list) so user uploads can be enabled later +-- without a schema migration. +-- +-- ★ IMPORTANT — TWO-TRANSACTION STRUCTURE ★ +-- Postgres requires `ALTER TYPE ... ADD VALUE` to be COMMITTED before +-- the new value can be USED. The Tag seed (section 18) inserts rows +-- using `'Model3D'::"TagTarget"` — if it runs in the same transaction +-- as the ALTER TYPE in section 1, you get: +-- ERROR: unsafe use of new value "Model3D" of enum type "TagTarget" +-- +-- This file is split into TWO transactions via the explicit +-- `COMMIT; BEGIN;` between sections 1 and 2. If your apply tool +-- (psql / Retool / etc.) wraps the whole file in BEGIN/COMMIT, that +-- still works: the inner COMMIT closes the wrapper, the inner BEGIN +-- starts a fresh transaction, and the outer COMMIT closes that one. +-- +-- If your tool runs each statement in autocommit mode, the COMMIT +-- and BEGIN below are effectively no-ops — still safe. +-- ============================================================ + +-- ----------------------------- +-- 1. AlterEnum: existing enums (transaction 1) +-- ----------------------------- +BEGIN; +ALTER TYPE "TagTarget" ADD VALUE IF NOT EXISTS 'Model3D'; +ALTER TYPE "CosmeticEntity" ADD VALUE IF NOT EXISTS 'Model3D'; +ALTER TYPE "CollectionType" ADD VALUE IF NOT EXISTS 'Model3D'; +ALTER TYPE "EntityType" ADD VALUE IF NOT EXISTS 'Model3D'; +COMMIT; + +-- ★ Transaction 1 committed. New enum values are now usable below. ★ +BEGIN; + +-- ----------------------------- +-- 2. CreateEnum +-- ----------------------------- +CREATE TYPE "Model3DStatus" AS ENUM ('Draft', 'Published', 'Unpublished', 'Deleted'); +CREATE TYPE "Model3DEngagementType" AS ENUM ('Favorite', 'Hide', 'Notify'); +-- Model3DFile.format is a free-text String (e.g. 'glb', 'fbx', 'obj', +-- 'usdz', 'stl') — matches civitai-client Model3dBlob.format. +-- Free-text instead of an enum so new formats from Meshy/orchestrator +-- can be ingested without a schema migration. + +-- ----------------------------- +-- 3. CreateTable: Model3DLicense +-- A separate table from `License` because physical-print / asset +-- licensing has dimensions (print-farm, redistribution) that the AI- +-- license shape doesn't cover. See plan §2.7. +-- ----------------------------- +CREATE TABLE "Model3DLicense" ( + "id" SERIAL NOT NULL, + "name" TEXT NOT NULL, + "description" TEXT NOT NULL, + "allowCommercialUse" BOOLEAN NOT NULL DEFAULT false, + "allowPrintFarm" BOOLEAN NOT NULL DEFAULT false, + "allowDerivatives" BOOLEAN NOT NULL DEFAULT true, + "allowRedistribution" BOOLEAN NOT NULL DEFAULT false, + "requireAttribution" BOOLEAN NOT NULL DEFAULT true, + "isCustom" BOOLEAN NOT NULL DEFAULT false, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "Model3DLicense_pkey" PRIMARY KEY ("id"), + CONSTRAINT "Model3DLicense_printfarm_requires_commercial" + CHECK (NOT "allowPrintFarm" OR "allowCommercialUse") +); + +CREATE UNIQUE INDEX "Model3DLicense_name_key" ON "Model3DLicense"("name"); + +-- ----------------------------- +-- 4. CreateTable: Model3D +-- v1 source = orchestrator generation (workflowId/sourceImageId set). +-- Schema is upload-ready: workflowId is nullable so a future "user +-- upload" flow can write rows with workflowId = NULL. +-- thumbnailImageId is nullable + SET NULL so existing image-deletion +-- paths don't throw on FK restriction. App layer enforces "thumbnail +-- required to publish". +-- ----------------------------- +CREATE TABLE "Model3D" ( + "id" SERIAL NOT NULL, + "name" CITEXT NOT NULL, + "description" TEXT, + "userId" INTEGER NOT NULL, + "thumbnailImageId" INTEGER, + "licenseId" INTEGER NOT NULL, + "licenseDetails" TEXT, -- free-text for custom licenses + + -- Generation provenance (NULL for future user-uploaded entries) + "workflowId" TEXT, -- orchestrator workflow ID + "sourceImageId" INTEGER, -- image-to-3D source + "generationParams" JSONB, -- PolyGen input snapshot (prompt, topology, polycount, seed, ...) + + "status" "Model3DStatus" NOT NULL DEFAULT 'Draft', + "nsfw" BOOLEAN NOT NULL DEFAULT false, + "tosViolation" BOOLEAN NOT NULL DEFAULT false, + "poi" BOOLEAN NOT NULL DEFAULT false, + "minor" BOOLEAN NOT NULL DEFAULT false, + "unlisted" BOOLEAN NOT NULL DEFAULT false, + "lockedProperties" TEXT[] NOT NULL DEFAULT ARRAY[]::TEXT[], + "availability" "Availability" NOT NULL DEFAULT 'Public', + "nsfwLevel" INTEGER NOT NULL DEFAULT 0, + "meta" JSONB NOT NULL DEFAULT '{}', + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "publishedAt" TIMESTAMP(3), + "deletedAt" TIMESTAMP(3), + "deletedBy" INTEGER, + + CONSTRAINT "Model3D_pkey" PRIMARY KEY ("id") +); + +-- ----------------------------- +-- 5. CreateTable: Model3DFile +-- A Model3D has 1..N files representing the SAME asset in different +-- formats (glb, fbx, obj, usdz, stl). The detail page presents these +-- as a "select format" dropdown, not as separate models. +-- No size cap in v1 (storage is not a constraint). Revisit if egress +-- costs spike post-launch. +-- ----------------------------- +CREATE TABLE "Model3DFile" ( + "id" SERIAL NOT NULL, + "model3dId" INTEGER NOT NULL, + "name" TEXT NOT NULL, + "url" TEXT NOT NULL, + "sizeKB" DOUBLE PRECISION NOT NULL, + "format" TEXT NOT NULL, -- 'glb', 'fbx', 'obj', 'usdz', 'stl', ... + "isPrimary" BOOLEAN NOT NULL DEFAULT false, -- the default viewer/download format + "metadata" JSONB, + "virusScanResult" "ScanResultCode" NOT NULL DEFAULT 'Success', -- orchestrator-trusted in v1; switch to 'Pending' when user uploads land + "virusScanMessage" TEXT, + "rawScanResult" JSONB, + "scannedAt" TIMESTAMP(3), + "scanRequestedAt" TIMESTAMP(3), + "exists" BOOLEAN, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "Model3DFile_pkey" PRIMARY KEY ("id") +); + +-- ----------------------------- +-- 6. CreateTable: TagsOnModel3D +-- ----------------------------- +CREATE TABLE "TagsOnModel3D" ( + "model3dId" INTEGER NOT NULL, + "tagId" INTEGER NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "TagsOnModel3D_pkey" PRIMARY KEY ("model3dId", "tagId") +); + +-- ----------------------------- +-- 7. CreateTable: Model3DEngagement +-- ----------------------------- +CREATE TABLE "Model3DEngagement" ( + "userId" INTEGER NOT NULL, + "model3dId" INTEGER NOT NULL, + "type" "Model3DEngagementType" NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "Model3DEngagement_pkey" PRIMARY KEY ("userId", "model3dId") +); + +-- ----------------------------- +-- 8. CreateTable: Model3DReport +-- ----------------------------- +CREATE TABLE "Model3DReport" ( + "model3dId" INTEGER NOT NULL, + "reportId" INTEGER NOT NULL, + + CONSTRAINT "Model3DReport_pkey" PRIMARY KEY ("reportId", "model3dId") +); + +-- ----------------------------- +-- 9. CreateTable: Model3DReview +-- Parallel to ResourceReview. Star rating + recommend boolean + details. +-- One review per (model3dId, userId). +-- Review-on-review reactions are NOT included in v1. +-- ----------------------------- +CREATE TABLE "Model3DReview" ( + "id" SERIAL NOT NULL, + "model3dId" INTEGER NOT NULL, + "userId" INTEGER NOT NULL, + "rating" INTEGER NOT NULL, + "recommended" BOOLEAN NOT NULL DEFAULT true, + "details" TEXT, + "nsfw" BOOLEAN NOT NULL DEFAULT false, + "tosViolation" BOOLEAN NOT NULL DEFAULT false, + "exclude" BOOLEAN NOT NULL DEFAULT false, + "metadata" JSONB, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "Model3DReview_pkey" PRIMARY KEY ("id"), + CONSTRAINT "Model3DReview_rating_check" CHECK ("rating" BETWEEN 1 AND 5) +); + +-- ----------------------------- +-- 10. CreateTable: Model3DReviewReport +-- ----------------------------- +CREATE TABLE "Model3DReviewReport" ( + "model3dReviewId" INTEGER NOT NULL, + "reportId" INTEGER NOT NULL, + + CONSTRAINT "Model3DReviewReport_pkey" PRIMARY KEY ("reportId", "model3dReviewId") +); + +-- ----------------------------- +-- 11. CreateTable: Model3DMetric +-- Denormalized columns at the bottom mirror ModelMetric so feed queries +-- can sort/filter without joining back to Model3D. +-- downloadCount is populated from ClickHouse events (not from a Postgres +-- DownloadHistory table — pilot intentionally avoids that table). +-- ----------------------------- +CREATE TABLE "Model3DMetric" ( + "model3dId" INTEGER NOT NULL, + "downloadCount" INTEGER NOT NULL DEFAULT 0, + "commentCount" INTEGER NOT NULL DEFAULT 0, + "collectedCount" INTEGER NOT NULL DEFAULT 0, + "imageCount" INTEGER NOT NULL DEFAULT 0, + "tippedCount" INTEGER NOT NULL DEFAULT 0, + "tippedAmountCount" INTEGER NOT NULL DEFAULT 0, + "ratingCount" INTEGER NOT NULL DEFAULT 0, + "ratingAvg" DOUBLE PRECISION NOT NULL DEFAULT 0, + "recommendedCount" INTEGER NOT NULL DEFAULT 0, + -- reactionCount denormalized from the thumbnail Image's ImageMetric; + -- feed sort "by popular" reads this column directly to avoid joining. + "reactionCount" INTEGER NOT NULL DEFAULT 0, + "earnedAmount" INTEGER NOT NULL DEFAULT 0, + "updatedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + -- Denormalized from Model3D for feed query perf + "nsfwLevel" INTEGER NOT NULL DEFAULT 0, + "userId" INTEGER NOT NULL DEFAULT 0, + "status" "Model3DStatus" NOT NULL DEFAULT 'Draft', + "availability" "Availability" NOT NULL DEFAULT 'Public', + "poi" BOOLEAN NOT NULL DEFAULT false, + "minor" BOOLEAN NOT NULL DEFAULT false, + + CONSTRAINT "Model3DMetric_pkey" PRIMARY KEY ("model3dId") +); + +-- ----------------------------- +-- 12. AlterTable: Thread — add model3dId +-- ----------------------------- +ALTER TABLE "Thread" ADD COLUMN "model3dId" INTEGER; +CREATE UNIQUE INDEX "Thread_model3dId_key" ON "Thread"("model3dId"); + +-- Reviews also get threads (for review comments later). +ALTER TABLE "Thread" ADD COLUMN "model3dReviewId" INTEGER; +CREATE UNIQUE INDEX "Thread_model3dReviewId_key" ON "Thread"("model3dReviewId"); + +-- ----------------------------- +-- 13. AlterTable: Post — add model3dId + model3dReviewId +-- model3dId = community "Makes/Uses" Posts + creator generation Post +-- model3dReviewId = Posts owned by a review (for review image attachments) +-- ----------------------------- +ALTER TABLE "Post" ADD COLUMN "model3dId" INTEGER; +ALTER TABLE "Post" ADD COLUMN "model3dReviewId" INTEGER; +CREATE INDEX "Post_model3dId_idx" ON "Post"("model3dId"); +CREATE UNIQUE INDEX "Post_model3dReviewId_key" ON "Post"("model3dReviewId"); + +-- ----------------------------- +-- 14. AlterTable: CollectionItem — add model3dId + replace unique index +-- The original unique index was created with CREATE UNIQUE INDEX (not ADD +-- CONSTRAINT) and its name was truncated by Postgres to 63 chars, ending +-- "modelI_key" (not "modelId_key"). Drop BOTH names defensively, then +-- create a new shorter-named index that includes model3dId. +-- See prisma/migrations/20230719152210_setup_for_collection_review_items/migration.sql:31 +-- ----------------------------- +ALTER TABLE "CollectionItem" ADD COLUMN "model3dId" INTEGER; + +DROP INDEX IF EXISTS "CollectionItem_collectionId_articleId_postId_imageId_modelI_key"; +DROP INDEX IF EXISTS "CollectionItem_collectionId_articleId_postId_imageId_modelId_key"; + +CREATE UNIQUE INDEX "CollectionItem_unique_entity_with_model3d_key" + ON "CollectionItem"("collectionId", "articleId", "postId", "imageId", "modelId", "model3dId"); + +CREATE INDEX "CollectionItem_model3dId_idx" ON "CollectionItem" USING HASH ("model3dId"); + +-- ----------------------------- +-- 15. Indexes for new tables +-- ----------------------------- +CREATE INDEX "Model3D_userId_status_publishedAt_idx" ON "Model3D"("userId", "status", "publishedAt" DESC); +CREATE INDEX "Model3D_status_publishedAt_idx" ON "Model3D"("status", "publishedAt" DESC); +CREATE INDEX "Model3D_status_nsfwLevel_publishedAt_idx" ON "Model3D"("status", "nsfwLevel", "publishedAt" DESC); +CREATE INDEX "Model3D_name_idx" ON "Model3D"("name"); +CREATE UNIQUE INDEX "Model3D_thumbnailImageId_key" ON "Model3D"("thumbnailImageId"); +CREATE INDEX "Model3D_licenseId_idx" ON "Model3D" USING HASH ("licenseId"); +CREATE UNIQUE INDEX "Model3D_workflowId_key" ON "Model3D"("workflowId"); -- prevents duplicate Post-from-Generation +CREATE INDEX "Model3D_sourceImageId_idx" ON "Model3D" USING HASH ("sourceImageId"); + +CREATE INDEX "Model3DFile_model3dId_idx" ON "Model3DFile" USING HASH ("model3dId"); +CREATE UNIQUE INDEX "Model3DFile_model3dId_format_key" ON "Model3DFile"("model3dId", "format"); +-- At most one isPrimary=true file per Model3D +CREATE UNIQUE INDEX "Model3DFile_model3dId_isPrimary_key" ON "Model3DFile"("model3dId") WHERE "isPrimary"; + +CREATE UNIQUE INDEX "Model3DReport_reportId_key" ON "Model3DReport"("reportId"); +CREATE INDEX "Model3DReport_model3dId_idx" ON "Model3DReport" USING HASH ("model3dId"); + +CREATE UNIQUE INDEX "Model3DReview_model3dId_userId_key" ON "Model3DReview"("model3dId", "userId"); +CREATE INDEX "Model3DReview_model3dId_idx" ON "Model3DReview" USING HASH ("model3dId"); +CREATE INDEX "Model3DReview_userId_idx" ON "Model3DReview" USING HASH ("userId"); + +CREATE UNIQUE INDEX "Model3DReviewReport_reportId_key" ON "Model3DReviewReport"("reportId"); +CREATE INDEX "Model3DReviewReport_model3dReviewId_idx" ON "Model3DReviewReport" USING HASH ("model3dReviewId"); + +CREATE INDEX "TagsOnModel3D_model3dId_idx" ON "TagsOnModel3D" USING HASH ("model3dId"); +CREATE INDEX "TagsOnModel3D_tagId_idx" ON "TagsOnModel3D" USING HASH ("tagId"); + +CREATE INDEX "Model3DEngagement_model3dId_idx" ON "Model3DEngagement" USING HASH ("model3dId"); + +-- ----------------------------- +-- 16. AddForeignKey +-- ----------------------------- +ALTER TABLE "Model3D" + ADD CONSTRAINT "Model3D_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE, + ADD CONSTRAINT "Model3D_deletedBy_fkey" FOREIGN KEY ("deletedBy") REFERENCES "User"("id") ON DELETE SET NULL ON UPDATE CASCADE, + ADD CONSTRAINT "Model3D_thumbnailImageId_fkey" FOREIGN KEY ("thumbnailImageId") REFERENCES "Image"("id") ON DELETE SET NULL ON UPDATE CASCADE, + ADD CONSTRAINT "Model3D_sourceImageId_fkey" FOREIGN KEY ("sourceImageId") REFERENCES "Image"("id") ON DELETE SET NULL ON UPDATE CASCADE, + ADD CONSTRAINT "Model3D_licenseId_fkey" FOREIGN KEY ("licenseId") REFERENCES "Model3DLicense"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +ALTER TABLE "Model3DFile" + ADD CONSTRAINT "Model3DFile_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +ALTER TABLE "TagsOnModel3D" + ADD CONSTRAINT "TagsOnModel3D_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE CASCADE ON UPDATE CASCADE, + ADD CONSTRAINT "TagsOnModel3D_tagId_fkey" FOREIGN KEY ("tagId") REFERENCES "Tag"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +ALTER TABLE "Model3DEngagement" + ADD CONSTRAINT "Model3DEngagement_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE, + ADD CONSTRAINT "Model3DEngagement_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +ALTER TABLE "Model3DReport" + ADD CONSTRAINT "Model3DReport_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE CASCADE ON UPDATE CASCADE, + ADD CONSTRAINT "Model3DReport_reportId_fkey" FOREIGN KEY ("reportId") REFERENCES "Report"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +ALTER TABLE "Model3DReview" + ADD CONSTRAINT "Model3DReview_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE CASCADE ON UPDATE CASCADE, + ADD CONSTRAINT "Model3DReview_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +ALTER TABLE "Model3DReviewReport" + ADD CONSTRAINT "Model3DReviewReport_model3dReviewId_fkey" FOREIGN KEY ("model3dReviewId") REFERENCES "Model3DReview"("id") ON DELETE CASCADE ON UPDATE CASCADE, + ADD CONSTRAINT "Model3DReviewReport_reportId_fkey" FOREIGN KEY ("reportId") REFERENCES "Report"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +ALTER TABLE "Model3DMetric" + ADD CONSTRAINT "Model3DMetric_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +ALTER TABLE "Thread" + ADD CONSTRAINT "Thread_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE SET NULL ON UPDATE CASCADE, + ADD CONSTRAINT "Thread_model3dReviewId_fkey" FOREIGN KEY ("model3dReviewId") REFERENCES "Model3DReview"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +ALTER TABLE "Post" + ADD CONSTRAINT "Post_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE SET NULL ON UPDATE CASCADE, + ADD CONSTRAINT "Post_model3dReviewId_fkey" FOREIGN KEY ("model3dReviewId") REFERENCES "Model3DReview"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +ALTER TABLE "CollectionItem" + ADD CONSTRAINT "CollectionItem_model3dId_fkey" FOREIGN KEY ("model3dId") REFERENCES "Model3D"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- ----------------------------- +-- 17. Seed: Model3DLicense templates +-- For `isCustom = true` rows, the boolean columns are advisory only — +-- the creator's free-text `Model3D.licenseDetails` is authoritative. +-- ----------------------------- +INSERT INTO "Model3DLicense" ("name", "description", "allowCommercialUse", "allowPrintFarm", "allowDerivatives", "allowRedistribution", "requireAttribution", "isCustom") VALUES + ('CC-BY 4.0', 'Creative Commons Attribution 4.0 — anyone may use commercially with attribution.', true, true, true, true, true, false), + ('CC-BY-NC 4.0', 'Creative Commons Attribution Non-Commercial 4.0 — non-commercial use with attribution.', false, false, true, true, true, false), + ('Personal Use Only', 'Free for personal use only. No commercial use, no redistribution.', false, false, false, false, true, false), + ('No Commercial Print Farm', 'Personal and commercial use allowed, but no commercial print-farm operations.', true, false, true, true, true, false), + ('All Rights Reserved', 'Default restrictive license. No use, derivatives, or redistribution without explicit permission.', false, false, false, false, true, false), + ('Custom', 'Custom license — see free-text licenseDetails on the model.', false, false, false, false, true, true) +ON CONFLICT ("name") DO NOTHING; + +-- ----------------------------- +-- 18. Seed: starter Tag taxonomy for Model3D +-- Generic 3D tags — applicable to printing, games, animation, viz. +-- For names that already exist, append 'Model3D' to their target array. +-- ----------------------------- +WITH starter_tags(name, is_category) AS ( + VALUES + -- Subject + ('character', true), + ('creature', true), + ('environment', true), + ('prop', true), + ('vehicle', true), + ('architecture', true), + ('furniture', true), + -- Style + ('low-poly', false), + ('stylized', false), + ('realistic', false), + ('abstract', false), + ('sci-fi', false), + ('fantasy', false) +) +INSERT INTO "Tag" ("name", "target", "type", "isCategory", "createdAt", "updatedAt") +SELECT name, ARRAY['Model3D']::"TagTarget"[], 'System', is_category, NOW(), NOW() +FROM starter_tags +ON CONFLICT ("name") DO UPDATE + SET "target" = CASE + WHEN 'Model3D' = ANY("Tag"."target") THEN "Tag"."target" + ELSE array_append("Tag"."target", 'Model3D'::"TagTarget") + END; + +COMMIT; +-- Transaction 2 committed. Migration complete. diff --git a/prisma/migrations/20260605120000_model3d_drop_legacy_rating/migration.sql b/prisma/migrations/20260605120000_model3d_drop_legacy_rating/migration.sql new file mode 100644 index 0000000000..d72645f09f --- /dev/null +++ b/prisma/migrations/20260605120000_model3d_drop_legacy_rating/migration.sql @@ -0,0 +1,19 @@ +-- Model3D reviews moved from 1-5 stars to thumbs up/down. The legacy +-- `rating` column on Model3DReview and the derived `ratingAvg` on +-- Model3DMetric are no longer written or read by any consumer — the UI +-- and the metrics rollup both derive everything from `recommendedCount` +-- and `ratingCount`. This migration drops both columns end-to-end. +-- +-- Safe to run because: +-- * No production deploy has shipped the original star UI. +-- * All upserts/selects/aggregations in the codebase have been +-- migrated to read `recommended` directly. +-- +-- Apply manually per the project's policy (we do NOT use `prisma migrate +-- deploy` — see CLAUDE.md "Database"). + +ALTER TABLE "Model3DReview" + DROP COLUMN IF EXISTS "rating"; + +ALTER TABLE "Model3DMetric" + DROP COLUMN IF EXISTS "ratingAvg"; diff --git a/prisma/migrations/20260612120000_model3d_gallery_settings/migration.sql b/prisma/migrations/20260612120000_model3d_gallery_settings/migration.sql new file mode 100644 index 0000000000..6326f9bd4c --- /dev/null +++ b/prisma/migrations/20260612120000_model3d_gallery_settings/migration.sql @@ -0,0 +1,15 @@ +-- Per-Model3D gallery moderation settings (creator/mod hide images, users, +-- tags). Mirrors `Model.gallerySettings` JSONB but without a version +-- dimension — a 3D model has no versions, so `images` is a flat array of +-- hidden imageIds rather than a per-version map. +-- +-- Shape: +-- { +-- "users": number[], -- hidden user ids +-- "tags": number[], -- hidden tag ids +-- "images": number[] -- hidden image ids +-- } +ALTER TABLE "Model3D" + ADD COLUMN IF NOT EXISTS "gallerySettings" JSONB + NOT NULL + DEFAULT '{"users":[],"tags":[],"images":[]}'::jsonb; diff --git a/prisma/migrations/20260617120000_model3dfile_variant/migration.sql b/prisma/migrations/20260617120000_model3dfile_variant/migration.sql new file mode 100644 index 0000000000..65cdaec048 --- /dev/null +++ b/prisma/migrations/20260617120000_model3dfile_variant/migration.sql @@ -0,0 +1,23 @@ +-- Model3DFile.variant — allows multiple Model3DFile rows to share a format +-- under the same Model3D. The PolyGen workflow can emit several semantically +-- distinct GLB exports for a single generation (textured base mesh, rigged +-- variant, animated variant, walking/running templates with armature-only +-- siblings); without a `variant` discriminator the existing +-- @@unique([model3dId, format]) constraint forces us to drop all but one. +-- +-- Values used by the polygen handler: +-- primary — the textured + remeshed base mesh +-- rigged — the rigged variant +-- animated — the animated variant +-- walking — basicAnimations.walkingModel / walkingFbxModel +-- walking-armature — basicAnimations.walkingArmatureModel +-- running — basicAnimations.runningModel / runningFbxModel +-- running-armature — basicAnimations.runningArmatureModel +ALTER TABLE "Model3DFile" + ADD COLUMN IF NOT EXISTS "variant" TEXT NOT NULL DEFAULT 'primary'; + +-- Move the uniqueness from (model3dId, format) to (model3dId, format, variant) +-- so each variant can carry its own glb + fbx pair. +DROP INDEX IF EXISTS "Model3DFile_model3dId_format_key"; +CREATE UNIQUE INDEX IF NOT EXISTS "Model3DFile_model3dId_format_variant_key" + ON "Model3DFile" ("model3dId", "format", "variant"); diff --git a/prisma/migrations/20260618120000_model3d_category_system_tag/migration.sql b/prisma/migrations/20260618120000_model3d_category_system_tag/migration.sql new file mode 100644 index 0000000000..ba7b479058 --- /dev/null +++ b/prisma/migrations/20260618120000_model3d_category_system_tag/migration.sql @@ -0,0 +1,11 @@ +-- Seed the system tag that anchors Model3D category tags. Mirrors the +-- existing `image category`, `model category`, `post category`, and +-- `article category` system tags — `getCategoryTags('model3d')` looks +-- for this row by name and treats every tag linked from it via +-- `TagsOnTags{type:Parent}` as a Model3D category. +-- +-- Idempotent: the unique partial index on `Tag.name` makes the +-- ON CONFLICT no-op safe to re-run. +INSERT INTO "Tag" ("name", "type", "target", "createdAt", "updatedAt") +VALUES ('model3d category', 'System', ARRAY[]::"TagTarget"[], NOW(), NOW()) +ON CONFLICT ("name") DO NOTHING; diff --git a/prisma/migrations/20260618130000_model3d_tag_target_backfill/migration.sql b/prisma/migrations/20260618130000_model3d_tag_target_backfill/migration.sql new file mode 100644 index 0000000000..9e267cbe8b --- /dev/null +++ b/prisma/migrations/20260618130000_model3d_tag_target_backfill/migration.sql @@ -0,0 +1,25 @@ +-- Backfill `Tag.target` so every tag currently attached to a Model3D +-- carries the `Model3D` target value. +-- +-- Why this is needed: +-- `upsertModel3D` originally reused existing tags by name (e.g. `pokemon`, +-- which was created with target `['Image']` for the image feed) and +-- inserted a `TagsOnModel3D` row, but it never updated the tag's own +-- `target` array. That left a population of tags that are attached to +-- Model3Ds but invisible to anything that filters tags by +-- `target && '{Model3D}'`: +-- - the Model3D tag picker autocomplete (`tag.getAll` with +-- `entityType: [TagTarget.Model3D]`) +-- - the search-index target filter +-- - category-tag discovery for the feed scroller +-- +-- `upsertModel3D` has been patched to append `Model3D` to the target +-- array on every attach (see model3d.service.ts), so this is a one-shot +-- backfill for tags that pre-date that patch. +-- +-- Idempotent: the `NOT ('Model3D' = ANY ...)` guard means rerunning this +-- on an already-clean dataset is a no-op. +UPDATE "Tag" t +SET "target" = t."target" || ARRAY['Model3D']::"TagTarget"[] +WHERE EXISTS (SELECT 1 FROM "TagsOnModel3D" tom WHERE tom."tagId" = t.id) + AND NOT ('Model3D' = ANY (t."target")); diff --git a/prisma/migrations/20260619120000_register_civitai_cli_oauth_client/migration.sql b/prisma/migrations/20260619120000_register_civitai_cli_oauth_client/migration.sql new file mode 100644 index 0000000000..3969235e7b --- /dev/null +++ b/prisma/migrations/20260619120000_register_civitai_cli_oauth_client/migration.sql @@ -0,0 +1,56 @@ +-- Register the first-party `civitai-cli` OAuth client. +-- +-- This is a PUBLIC (no-secret) client used by the official `civitai` CLI to log +-- in via the OAuth device-authorization grant (RFC 8628) and obtain a token that +-- the App Blocks submit endpoint (api/v1/blocks/submit-version) accepts. The CLI +-- hardcodes the stable client id `civitai-cli`. +-- +-- allowedScopes = TokenScope.UserRead | TokenScope.AppBlocksSubmit +-- = 1 | 33554432 +-- = 33554433 +-- (AppBlocksSubmit, bit 25 = 1<<25 = 33554432, is opt-in and INTENTIONALLY +-- excluded from TokenScope.Full = 33554431. This client is the only first-party +-- consumer that requests it.) +-- +-- Owner: the civitai system account (User id -1) — the first-party convention. +-- +-- IDEMPOTENT: ON CONFLICT DO NOTHING on the PK so re-applying is a no-op. The +-- WHERE EXISTS guard avoids an FK violation if the civitai system User row is +-- absent in a given environment (in that case this inserts nothing and the row +-- must be created manually with a valid owner userId). +-- +-- ⚠️ MANUAL-APPLY: per the cluster ops rule, civitai DB migrations are NOT +-- auto-applied. A human applies this to prod (CNPG nvme0) and the dev clone. +INSERT INTO "OauthClient" ( + "id", + "secret", + "name", + "description", + "logoUrl", + "redirectUris", + "allowedOrigins", + "grants", + "allowedScopes", + "isConfidential", + "userId", + "isVerified", + "createdAt", + "updatedAt" +) +SELECT + 'civitai-cli', + NULL, + 'Civitai CLI', + 'Official Civitai command-line tool. Used to submit App Blocks for review and manage your apps from the terminal.', + NULL, + ARRAY['http://127.0.0.1/callback', 'http://localhost/callback']::TEXT[], + ARRAY[]::TEXT[], + ARRAY['authorization_code', 'refresh_token', 'urn:ietf:params:oauth:grant-type:device_code']::TEXT[], + 33554433, + false, + -1, + true, + CURRENT_TIMESTAMP, + CURRENT_TIMESTAMP +WHERE EXISTS (SELECT 1 FROM "User" WHERE "id" = -1) +ON CONFLICT ("id") DO NOTHING; diff --git a/prisma/migrations/20260621120000_app_block_reviews/migration.sql b/prisma/migrations/20260621120000_app_block_reviews/migration.sql new file mode 100644 index 0000000000..aa927db27e --- /dev/null +++ b/prisma/migrations/20260621120000_app_block_reviews/migration.sql @@ -0,0 +1,54 @@ +-- ============================================================ +-- App Blocks — MARKETPLACE REVIEWS (5-star) +-- ============================================================ +-- A parallel table to "ResourceReview" (which is hard-bound to model/version +-- FKs) so app blocks can carry star ratings without bending the model-review +-- shape. Backs: +-- - blocks.upsertReview / blocks.listReviews (gated dark behind the +-- mod-segmented appBlocks Flipt flag) +-- - getAppRatingTotals (AVG/COUNT, excludes `exclude` rows AND self-reviews) +-- - the marketplace `rating` sort (Bayesian shrinkage) + avg/count on cards +-- - the blue-buzz "leave a review" reward (fires ONCE per (user, app) on the +-- first create; per-(user,app) dedup anchored on the unique constraint) +-- +-- ⚠️ MANUAL-APPLY. The main civitai DB (CNPG nvme0, role=postgres) does NOT run +-- `prisma migrate deploy` — this file is committed for HISTORY only and applied +-- BY HAND (psql) per environment. Apply to: +-- 1. prod nvme0 (the live civitai DB) +-- 2. the dev clone (cnpg-cluster-dev, ns cnpg-database-dev) +-- The table is ADDITIVE + read only by the dark, mod-gated marketplace + +-- review procedures, so applying it ahead of the code deploy is inert. +-- +-- IF-NOT-EXISTS guards are used so a manual re-run is a no-op (Prisma's own +-- DDL is not idempotent; this is hand-applied, so we make it safe to re-run). + +CREATE TABLE IF NOT EXISTS "app_block_reviews" ( + "id" SERIAL PRIMARY KEY, + -- The app block being reviewed. CASCADE so deleting an app reaps its reviews. + "app_block_id" TEXT NOT NULL REFERENCES "app_blocks"("id") ON DELETE CASCADE, + -- The reviewer. CASCADE on GDPR user-delete. + "user_id" INTEGER NOT NULL REFERENCES "User"("id") ON DELETE CASCADE, + -- 1..5 stars. Range is validated in the service (kept off the DB so a future + -- product change doesn't require a manual ALTER on a hand-applied table), but + -- a CHECK is cheap insurance against a bad direct write. + "rating" INTEGER NOT NULL, + "recommended" BOOLEAN NOT NULL DEFAULT true, + "details" TEXT, + -- Moderator controls. exclude / tos_violation keep abusive reviews out of the + -- rating aggregate + the Bayesian marketplace sort. + "exclude" BOOLEAN NOT NULL DEFAULT false, + "tos_violation" BOOLEAN NOT NULL DEFAULT false, + "created_at" TIMESTAMPTZ NOT NULL DEFAULT now(), + "updated_at" TIMESTAMPTZ NOT NULL DEFAULT now(), + + CONSTRAINT "app_block_reviews_rating_range" CHECK ("rating" >= 1 AND "rating" <= 5) +); + +-- One review per (user, app). Also the per-(user,app) idempotency anchor for the +-- blue-buzz reward (the create branch only fires when this insert succeeds). +CREATE UNIQUE INDEX IF NOT EXISTS "app_block_reviews_app_user_uniq" + ON "app_block_reviews" ("app_block_id", "user_id"); + +-- Aggregate read path: AVG(rating) / COUNT(*) WHERE NOT exclude, per app. +CREATE INDEX IF NOT EXISTS "app_block_reviews_app_agg_idx" + ON "app_block_reviews" ("app_block_id", "exclude"); diff --git a/prisma/migrations/20260621120000_bus_app_block_created_analytics_idx/migration.sql b/prisma/migrations/20260621120000_bus_app_block_created_analytics_idx/migration.sql new file mode 100644 index 0000000000..edc7d6e15f --- /dev/null +++ b/prisma/migrations/20260621120000_bus_app_block_created_analytics_idx/migration.sql @@ -0,0 +1,23 @@ +-- ============================================================ +-- App Blocks — author analytics (Phase 0) supporting index +-- ============================================================ +-- The author analytics dashboard's installs time-series groups +-- block_user_subscriptions by (app_block_id, created_at) for the apps the +-- caller owns. The existing indexes on this table are either partial +-- (scope='publisher_all_my_models' / 'viewer_personal') or keyed on +-- (user_id, ...) — none serve an "installs over time for THIS app" scan. +-- +-- This composite index makes that bounded: the dashboard always filters by +-- a small set of owned app_block ids + a clamped date range. Cheap to add +-- (the analytics read path is dark behind the appBlocks flag). +-- +-- ⚠️ MANUAL APPLY: CNPG nvme0 (main civitai DB) does NOT auto-apply Prisma +-- migrations. Apply this by hand per environment. On prod prefer: +-- CREATE INDEX CONCURRENTLY "bus_app_block_created_idx" +-- ON "block_user_subscriptions" ("app_block_id", "created_at" DESC); +-- (CONCURRENTLY can't run inside Prisma's migration transaction, hence the +-- plain form below for the migration history; run the CONCURRENTLY variant +-- against prod to avoid a write lock.) + +CREATE INDEX IF NOT EXISTS "bus_app_block_created_idx" + ON "block_user_subscriptions" ("app_block_id", "created_at" DESC); diff --git a/src/components/AppBlocks/AppAnalyticsPanel.tsx b/src/components/AppBlocks/AppAnalyticsPanel.tsx new file mode 100644 index 0000000000..9eebd5c53e --- /dev/null +++ b/src/components/AppBlocks/AppAnalyticsPanel.tsx @@ -0,0 +1,332 @@ +import { + Alert, + Badge, + Card, + Group, + Loader, + Select, + SimpleGrid, + Stack, + Table, + Text, + Title, + Tooltip, +} from '@mantine/core'; +import type { ChartOptions } from 'chart.js'; +import { + CategoryScale, + Chart as ChartJS, + Filler, + LinearScale, + LineElement, + PointElement, + Tooltip as ChartTooltip, +} from 'chart.js'; +import { IconInfoCircle } from '@tabler/icons-react'; +import { useMemo, useState } from 'react'; +import { Line } from 'react-chartjs-2'; +import { useComputedColorScheme, useMantineTheme } from '@mantine/core'; +import dayjs from '~/shared/utils/dayjs'; +import { trpc } from '~/utils/trpc'; + +ChartJS.register(CategoryScale, LinearScale, PointElement, LineElement, ChartTooltip, Filler); + +type TimePoint = { bucket: string; value: number }; + +type AnalyticsData = { + range: { from: string | Date; to: string | Date; granularity: 'day' | 'week' }; + notOwned: boolean; + installs: { total: number; active: number; series: TimePoint[] }; + runs: { count: number; buzzSpent: number; series: TimePoint[] }; + buzzPurchased: { count: number; buzzAmount: number; grossCents: number }; + engagement: { + apiCalls: number; + activeUsers: number; + errorRate: number; + topScopes: Array<{ scope: string; count: number }>; + topEndpoints: Array<{ endpoint: string; count: number }>; + }; +}; + +type MyApp = { + id: string; + appName: string | null; + blockId: string; +}; + +function MetricCard({ + label, + value, + sub, + tooltip, +}: { + label: string; + value: string; + sub?: string; + tooltip?: string; +}) { + return ( + + + + {label} + + {tooltip && ( + + + + )} + + + {value} + + {sub && ( + + {sub} + + )} + + ); +} + +function MiniLineChart({ + points, + granularity, +}: { + points: TimePoint[]; + granularity: 'day' | 'week'; +}) { + const theme = useMantineTheme(); + const colorScheme = useComputedColorScheme('dark'); + const lineColor = theme.colors.blue[colorScheme === 'dark' ? 5 : 6]; + + const data = useMemo( + () => ({ + labels: points.map((p) => p.bucket), + datasets: [ + { + data: points.map((p) => p.value), + borderColor: lineColor, + backgroundColor: `${lineColor}1f`, + borderWidth: 2, + fill: true, + tension: 0.3, + pointRadius: 0, + pointHoverRadius: 4, + pointHoverBackgroundColor: lineColor, + }, + ], + }), + [points, lineColor] + ); + + const options = useMemo>( + () => ({ + responsive: true, + maintainAspectRatio: false, + interaction: { mode: 'index', intersect: false }, + scales: { + x: { display: false }, + y: { display: false, beginAtZero: true }, + }, + plugins: { + legend: { display: false }, + title: { display: false }, + tooltip: { + displayColors: false, + callbacks: { + title: (items) => + dayjs.utc(items[0]?.label).format(granularity === 'week' ? 'MMM D' : 'MMM D'), + label: (item) => `${item.parsed.y}`, + }, + }, + }, + }), + [granularity] + ); + + if (!points.length) { + return ( + + No data in this range. + + ); + } + + return ( +
+ +
+ ); +} + +export function AppAnalyticsPanel() { + const { data: appsRaw, isLoading: appsLoading } = trpc.blocks.getMyApps.useQuery(); + const apps = (appsRaw as MyApp[] | undefined) ?? []; + const [appBlockId, setAppBlockId] = useState(null); + + const { + data: analyticsRaw, + isLoading: analyticsLoading, + error, + } = trpc.blocks.getMyAppAnalytics.useQuery({ + appBlockId: appBlockId ?? undefined, + }); + const analytics = analyticsRaw as AnalyticsData | undefined; + + const appOptions = [ + { value: '', label: 'All my apps' }, + ...apps.map((a) => ({ + value: a.id, + label: a.appName ?? a.blockId ?? a.id, + })), + ]; + + return ( + + +