* feat(cosmetics): perceptual-hash cosmetic artwork Cosmetic art never becomes an Image row, so it has never been hashed — the only duplicate check is a sha256 on the shop item, which a re-encode defeats. Hash it directly instead: the orchestrator's mediaHash step stands alone, so a one-step workflow submitted with `wait` returns a perceptual hash synchronously with no Image row and no scan webhook. Hashing is fire-and-forget after the write. A cosmetic must land whether or not the orchestrator answers, and rows left NULL — whether never attempted or failed — are swept by the backfill endpoint. Creator submissions still bypass this; they write Cosmetic inline in a transaction and are handled with submit-time matching in phase 2. Migration is committed but NOT applied. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(cosmetics): make the pHash migration re-runnable Migrations here are applied by hand per environment, so a re-run should no-op rather than error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(cosmetics): re-hash when artwork changes, not when metadata does Five review findings: Metadata-only edits re-submitted a workflow. `previous` is only loaded when `data` is present, so renaming a badge compared against undefined and always "changed". Replacing artwork left a stale hash the backfill never revisited. Several paths swap `data.url` with a raw update, and a row holding the previous image's hash is worse than holding none — matching would go looking for artwork that is no longer there. `pHashUrl` records what was hashed; the backfill re-sweeps wherever it disagrees with `data.url`, which self-heals regardless of who wrote the url. Product badges were never hashed on write, and with no cron they would have stayed NULL until someone ran the endpoint by hand. One Prisma failure aborted the whole backfill batch, discarding the tally for every row already done. A 1000-row default outlived the request that started it; the mod saw a dropped connection and would re-run against rows still in flight. Also: abort a submit the orchestrator accepts and abandons, keep the cosmetic id out of a new Axiom column (civitai-prod is at its field cap), and skip empty-string urls. Migration is committed but NOT applied. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(cosmetics): correct dryRun parsing, zero hashes, and backfill starvation `z.coerce.boolean()` is Boolean(value), so `?dryRun=false` asked for a real run and got a dry one — the operator sees a plausible pending count and no work done, with nothing to indicate why. `0n` is a legitimate perceptual hash (solid-colour artwork), and cosmetics are a corpus where flat frames are ordinary. A falsy check recorded it as a failure, leaving the row NULL and re-attempted by every later run. The backfill had no cursor, so rows that can never be hashed — artwork that 404s on the CDN — held the same head-of-queue slots on every invocation and stranded everything behind them. Added `afterId`, and `lastId` in the response to drive it. Also: warn on the docblock that these hashes are comparable only to other mediaHash output, not to legacy Image.pHash rows, which came from a different algorithm and compare as noise with no error to signal it. Share the one hash conversion rather than keeping two copies. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(cosmetics): move the pHash backfill to a one-off script A mod-only HTTP endpoint is permanent surface area for a temporary job, and it can't be driven without a browser session — which is why the backfill hadn't been run yet. The script requires an explicit --target and prints the database it resolved before touching anything; there is no default. Arguments are parsed strictly and an unrecognised one aborts, since a mistyped flag quietly doing the opposite of what was asked is what this class of bug looks like. Its cursor advances past every row it fetches, so artwork that can never be hashed no longer holds the head of the queue. If a recurring drift sweep is wanted later, that belongs in a job under src/server/jobs, not an endpoint. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(cosmetics): correct the backfill invocation and name the lane The documented command didn't work: npm swallows --target as its own config option, so the script aborted asking for a target it had been given. Documents the pnpm exec form, which was verified verbatim. --target selects the database and nothing else. The orchestrator lane is separate, and hashes from different lanes aren't comparable, so a dev lane writing prod rows would look successful and mean nothing. It's now in the banner rather than left implicit. Also notes that the `Using PROD database.` line above the banner comes from importing the db client and ignores --target, and that both targets are localhost tunnels distinguished only by port. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(cosmetics): load env before the hashing service in the backfill Every row the backfill submitted failed. ~/env/client snapshots process.env.NEXT_PUBLIC_* when it is imported; Next inlines those at build, but a tsx process does not, and ~/env/server calls dotenv in its module body — after its own import of ~/env/client has already been evaluated. NEXT_PUBLIC_IMAGE_LOCATION came out undefined, and getEdgeUrl drops a falsy base from its join rather than failing, so the orchestrator received a relative path and could not fetch it. The script now loads dotenv before importing the hashing service. getPerceptualHash also refuses a media url that isn't absolute, naming the missing variable. Without that the failure costs one orchestrator round-trip per row and leaves nothing to read afterwards — the whole corpus fails and looks like a slow run. Verified: hashes now returned in ~300-400ms and match the values the orchestrator returns for the same artwork when called directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(cosmetics): keep the hash helpers out of the search-index graph Three suites failed to load. cosmetic.service imports ~/server/search-index, which reaches meilisearch/client — a module that calls pLimit(env...) and registers prom collectors at import. Importing cosmetic.service from cosmetic-shop.service and product-badge.service pulled all of that into two graphs that never had it, past mocks that name modules by hand. The hash helpers never needed that graph: they want dbWrite, logToAxiom and getPerceptualHash. Moving them to their own module drops the edge, which fixes prepaid-membership-jobs with no test change at all. cosmetic-shop.service still reaches prom/client legitimately, via orchestrator.service -> workflows -> orchestrator-read-metrics, so those two mocks now spread the real module with importOriginal instead of listing exports. A hand-listed mock is coupled to the whole transitive graph of the thing under test, and nothing warns you when that graph grows. Verified like-for-like: 3/3 fail before, 11 suites / 53 tests pass after, matching the same command run against the branch base. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: prefer importOriginal over hand-listed vi.mock exports Twice in one day a service import dragged a load-time module into a suite's graph and the hand-listed mock no longer covered it. Typecheck and lint stay green, so only CI catches it. Also records the half that's easy to lose: a failing suite may be saying the code pulled in a dependency it doesn't want, not that the mock is too narrow. Widening the mock would have hidden that. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
18 KiB
Civitai Development Guide
How to work with us
We use markdown documents to discuss plans. Documentation goes in the docs/ folder.
Inline Comments
Occasionally, we comment back and forth as we make plans. Comments from us, are marked with @dev: and you can leave comments as well with @ai:. Please make comments inline in the document. If there are actions are requested in my comments, please take them.
New Comment Marking: When you add new comments, use an asterisk after the mention (e.g., @justin:* or @meta:*). Once you reply or acknowledge a comment, remove the asterisk so that I know it's been seen. Note: Sometimes I might forget to add the asterisk to my new comments, so please check all comments regardless of marking.
Example
@dev: This comment has been processed (asterisk removed)
@ai: Of course
@dev:* This is a new comment that needs attention
Tech Stack Overview
Core Technologies
- Framework: Next.js 14 with TypeScript
- UI Library: Mantine v7
- Styling: Tailwind CSS + SCSS Modules
- Database: PostgreSQL with Prisma ORM
- API: tRPC
- State Management: Zustand
- Authentication: NextAuth
- Search: Meilisearch
- Image Processing: Sharp
Additional Libraries
- React Query (Tanstack Query) for data fetching
- React Hook Form with Zod validation
- Tiptap for rich text editing
- Chart.js for data visualization
- Stripe/Paddle/PayPal for payments
Build Commands
Development
Always use the /dev-server skill to manage dev servers. Never use pnpm run dev directly.
Build & Deploy
pnpm run build # Production build
Code Quality
pnpm run typecheck # Run TypeScript type checking
pnpm run lint # Run ESLint
pnpm run prettier:check # Check Prettier formatting
pnpm run prettier:write # Auto-fix Prettier formatting
Testing
pnpm test # Run Playwright tests
pnpm run test:ui # Run tests with UI
Never put unit tests under src/pages
Next.js 16 treats every .ts/.tsx file under src/pages (incl. nested __tests__/) as a route, and next build runs a route-type validator over it. A Vitest test file there fails the build with Type '...test' does not satisfy the constraint 'ApiRouteConfig'. Property 'default' is missing — and only next build catches it: pnpm typecheck, pnpm test/vitest, and the CI typecheck/unit/component tasks all pass, so it sneaks through to the preview build-image step. Keep handler tests in a __tests__/ dir outside src/pages (e.g. src/server/__tests__/) and import the handler via the ~/pages/... alias. (Bit us on PR #2653.)
Prefer importOriginal over hand-listed vi.mock exports
A vi.mock that lists exports by hand couples the test to the entire transitive import graph of the thing under test, and nothing warns you when that graph grows. Adding one service import can drag in a module that builds pLimit/prom collectors at load (e.g. ~/server/search-index → meilisearch/client), and the suite then fails to load with an error far from the change — pnpm typecheck and pnpm lint stay green, so only CI catches it. Spread the real module and override only what you need:
vi.mock('~/server/prom/client', async (importOriginal) => ({
...(await importOriginal<typeof PromClient>()),
dbReadFallbackCounter: { inc: vi.fn() },
}));
Use a top-level import type * as PromClient — an inline typeof import('...') trips consistent-type-imports.
Before widening a mock, check whether the import edge is needed at all. A failing suite may be telling you the code pulled in a dependency it doesn't want, not that the mock is too narrow, and widening it would hide that. (Bit us twice in one day, Aug 2026, on two branches; one of those three suites was fixed by extracting the helpers into their own module instead.)
Database
pnpm run db:migrate:empty # Create an empty migration file
CRITICAL: We do NOT use prisma migrate deploy. Migrations are applied manually.
- Migration files in
packages/civitai-db-schema/prisma/migrations/exist for review/history but are never auto-run. That is the only directory Prisma reads — theprisma/migrations/path at the repo root predates the monorepo, no longer exists, and CI blocks re-creating it. - Each environment's DB is updated by a human running the SQL directly (psql, retool, etc.)
- The
_prisma_migrationstable is not the source of truth — do not rely on it - When you add a new migration: write the SQL, commit it, and surface to the user that it needs to be applied manually to wherever they want it (preview / staging / prod)
- Never suggest
prisma migrate deploy,prisma migrate resolve, or any auto-apply path
Release (requires user permission)
pnpm run release # Patch release (0.0.x) - default
pnpm run release:minor # Minor release (0.x.0)
pnpm run release:major # Major release (x.0.0)
IMPORTANT: Never run release commands without explicit user approval. These commands bump the version, push tags, and rebase the release branch.
Server-Side Architecture Map
src/server/ holds the most-edited (and largest) code in the repo. Read the specific file before changing it — several are huge, so grep within them rather than reading end-to-end (services/image.service.ts is ~7.9K lines).
- tRPC API —
trpc.ts(root router + procedure helpers),createContext.ts,middleware.trpc.ts,routers/(~93 per-domain routers),controllers/,schema/(zod input contracts),selectors/(Prismaselectfragments). - Images —
services/image.service.ts(~7.9K lines; the hot feed path —getInfiniteImages,getAllImages, NSFW/own-content merge). API surfacesrc/pages/api/v1/images/index.ts; index syncsearch-index/images.search-index.ts. - Models —
services/model.service.ts,search-index/models.search-index.ts. - Search (Meilisearch) —
meilisearch/client.ts(tags requests withX-Search-Actor),meilisearch/cleanup.ts,search-index/base.search-index.ts(shared sync engine). - Redis / caching —
redis/client.ts(clients incl. sysRedis),redis/caches.ts(createCachedObjectdefs + TTLs, e.g.imageMetaCache,tagIdsForImagesCache),utils/cache-helpers.ts. - Orchestrator (generation) —
orchestrator/get-orchestrator-token.ts(getOrchestratorToken),services/orchestrator/orchestrator.service.ts. - Auth —
auth/next-auth-options.ts,auth/session-user.ts,auth/token-refresh.ts. - Jobs (cron) —
jobs/job.ts(runner) + individual jobsjobs/*.ts(e.g.entity-moderation.ts,search-index-sync.ts). - Metrics / analytics —
metrics/*.metrics.ts(ClickHouse-backed entity metrics),clickhouse/. - DB —
db/db-helpers.ts(raw pg-pool config:connectionTimeoutMillis, labeled pool gauges), Prisma client; schemaprisma/schema.prisma. Migrations are applied manually — see the Database rule above. - Telemetry —
src/instrumentation.node.ts(OTEL: Prisma/Redis/HTTP auto-instrumentation + customwithSpan()fromutils/otel-helpers.ts),schema/track.schema.ts(ClickHouse action/event tags),prom/client.ts. - Health —
src/pages/api/health.tsruns sub-checks underPromise.all; a single slow check (e.g.searchMetrics) can exceed the kubelet probe budget.HEALTHCHECK_TIMEOUTenv gates it. - Other server domains —
games/(new-order/ratings),webhooks/,paddle/+coinbase/(payments),notifications/,signals/,rewards/; S3 helpers atsrc/utils/s3-utils.ts.
Component Standards
File Structure
src/
├── components/ # React components
│ ├── ComponentName/ # Component folder
│ │ ├── ComponentName.tsx
│ │ ├── ComponentName.module.scss # Optional SCSS module
│ │ └── utils.ts # Component utilities
├── hooks/ # Custom React hooks
├── server/ # Server-side code
├── utils/ # Shared utilities
└── store/ # Zustand stores
Component Patterns
1. Mantine Components
import { Button, Group, Text } from '@mantine/core';
import { IconBolt } from '@tabler/icons-react';
2. Tailwind Classes with clsx
import clsx from 'clsx';
<div className={clsx('flex items-center gap-2', conditionalClass && 'bg-blue-500')} />
3. SCSS Modules (when needed)
import styles from './Component.module.scss';
<div className={styles.container} />
4. TypeScript Patterns
- Use type imports when possible:
import type { ButtonProps } from '@mantine/core' - Define Props interfaces for components
- Use enums from
~/shared/utils/prisma/enums
Coding Standards
Imports Order
- External libraries (React, Mantine, etc.)
- Internal components (~/components/...)
- Hooks (~/hooks/...)
- Server/API code (~/server/...)
- Utils and helpers (~/utils/...)
- Types and enums
- Styles
State Management
- Use Zustand for global state
- Use React Query for server state
- Use React Hook Form for forms
API Calls
import { trpc } from '~/utils/trpc';
const { data, isLoading } = trpc.user.getProfile.useQuery();
Authentication
import { useCurrentUser } from '~/hooks/useCurrentUser';
const currentUser = useCurrentUser();
Comments
Comments are not type-checked, so they rot silently and become misleading. Write the minimum comment needed and bias toward none.
- Default to no comment. If the code is clear on its own, leave it alone. Prefer a clearer name, smaller method, or better type over a comment that explains confusing code.
- Only comment the non-obvious why: a rationale, tradeoff, gotcha, invariant, or workaround that the reader cannot recover from the code itself. Link an issue/PR when relevant.
- Never narrate the what. No comments that restate the next line, label obvious steps (
// loop over items), or describe what a well-named symbol already says. - Don't describe nearby code's current behavior (e.g. "this gates on X so Y happens"). That is exactly what goes stale when the other code changes. Comment the surprising fact, not the mechanics.
- No process/banner noise: no change-log narration (
// added to fix...), no "I changed X", no section-divider banners, no commented-out code. - When you do comment, keep it to a line or two. A long block almost always means the code or naming should be clearer instead.
Explain decisions in your response, not in the file. Rationale for a choice you just made — why you picked this shape, what you deliberately left out, what you considered and rejected — belongs in your reply to us, where we're already reading it. A comment justifying your work to a reviewer is the single most common way this section gets violated. If you catch yourself writing something you'd also say in chat, say it in chat only.
Comment in a separate pass. Write the code first with no comments, then reread it and add back only what's needed. Comments written while authoring never get evaluated — the reasoning is fresh, so it feels non-obvious when it isn't. Judge them against code you're reading, not code you're writing.
The keep test. For every comment that survives, you should be able to name the specific future edit that goes wrong without it. If the answer is "it's helpful context" or "it explains why this is correct," delete it. Being unable to name the failure means the code already says it — or should.
Clean up as you go. When you edit code that already has stale, redundant, or what-narrating comments, delete or fix them — don't preserve them just because they were there. The repo already has many such comments (a lot of them mine); treat touching nearby code as license to remove the noise, but keep edits scoped to what you're already working on rather than going on a separate comment-cleanup sweep.
Environment Setup
Required Environment Variables
- Database connection strings
- Authentication providers
- S3/CloudFlare credentials
- Payment provider keys
- Search service endpoints
Local Development
- Install dependencies:
pnpm install - Generate Prisma client:
pnpm run db:generate - Start dev server: Use
/dev-serverskill
Git Worktrees
When you create a new worktree (git worktree add …), always initialize the event-engine-common submodule
in it: git submodule sync --recursive && git submodule update --init event-engine-common.
Worktrees don't check out submodules automatically,
and without it pnpm typecheck/build fail with a wall of Cannot find module '.../event-engine-common/...'
errors (and the missing types cascade into unrelated implicitly has an 'any' type errors) — noise that looks
like your change broke something when it didn't.
Important Notes
- Read the full file before editing. Plan all changes, then make ONE complete edit. If you've edited a file 3+ times, stop and re-read the user's requirements.
- When the user corrects you, stop and re-read their message. Quote back what they asked for and confirm before proceeding.
- Every few turns, re-read the original request to make sure you haven't drifted from the goal.
- Act sooner. Don't read more than 3-5 files before making a change. Get a basic understanding, make the change, then iterate.
- When stuck, summarize what you've tried and ask the user for guidance instead of retrying the same approach.
- Re-read the user's last message before responding. Follow through on every instruction completely.
- After 2 consecutive tool failures, stop and change your approach entirely. Explain what failed and try a different strategy.
Performance
- Use dynamic imports for heavy components
- Implement virtual scrolling for large lists
- Optimize images with Next.js Image component
Security
- Never commit secrets or API keys
- Use environment variables
- Sanitize user input with sanitize-html
- Follow authentication best practices
Before Committing
- Run type checking:
pnpm run typecheck - Run linting:
pnpm run lint - Format code:
pnpm run prettier:write - Test changes locally
Stacked PRs — don't
- NEVER use stacked PRs — base every PR directly on the integration branch (
main, or a feature integration branch likefeat/...), never on another open PR's branch. Stacked PRs silently mis-merge: a squash-merged parent doesn't retarget the child, so the child lands on the orphaned parent branch instead of the real base and its changes go missing. - If a change depends on an unmerged PR, wait for that PR to merge, then branch off the updated base — or fold both changes into a single PR.
- (Bit us 2026-06-13: PR #2520's App Blocks W11 F5 was stacked on #2518 (F6) → #2520 squash-merged into the #2518 branch instead of
feat/app-blocks-main-v1; corrected via #2525.)
Common Patterns
Infinite Scroll
Use MasonryGrid or virtual scrolling components with React Query infinite queries.
Modals
Use Mantine modals with proper accessibility and keyboard handling.
Dialog Registry System
The project uses a dialog-registry system for managing modals:
- Register dialogs in
src/components/Dialog/dialog-registry.tsordialog-registry2.ts - Use
DialogProviderfor context-based modal management RoutedDialogProviderfor URL-based modal state- Access dialogs through the registry for consistent modal handling across the app
Forms
Use React Hook Form with Zod schemas for validation.
File Uploads
Use the S3 upload hooks and providers in the codebase.
Image Handling
Use EdgeImage component for optimized image loading with CDN support.
Debug Endpoints (src/pages/api/testing/*)
src/pages/api/testing/*.ts is the convention for hidden debug endpoints. Each endpoint is guarded by WEBHOOK_TOKEN (via WebhookEndpoint(...), which checks the ?token= query param) and exposes a handful of POST actions for experimenting with a feature without paying real money or hand-editing the DB.
To use one: read the endpoint's source file directly — the top-of-file comment documents the available actions and required params, and the zod schema is the authoritative contract. Agents should never need a wrapper skill; cURL with ?token=$WEBHOOK_TOKEN appended to the URL is enough.
When adding a new debug endpoint:
- Drop it at
src/pages/api/testing/<feature>.ts - Use
WebhookEndpoint(handler)for auth - Lead the file with a block comment listing each action + its params + a one-line description (see
src/pages/api/testing/referrals.tsfor the pattern) - Scope every destructive action to a single
userId/refereeIdper call so a misuse can't cascade
Feature Documentation
Feature-specific documentation lives in docs/features/. Before implementing a feature, check if documentation exists:
Core Systems Reference
| System | Documentation |
|---|---|
| Image Resources | docs/features/image-resources.md |
| NSFW Filtering | docs/features/nsfw-filtering.md |
| Buzz Accounts | docs/features/buzz-accounts.md |
| Notifications | docs/features/notifications.md |
| Metrics/Analytics | docs/features/metrics-analytics.md |
| Bitwise Flags | docs/features/bitwise-flags.md |
| Civitai LLM Client | docs/features/civitai-llm-client.md |
| Challenge Platform | docs/features/challenge-platform.md |
Troubleshooting
Memory Issues
Use cross-env NODE_OPTIONS with increased memory:
pnpm run dev-debug # Includes --max_old_space_size=8192
Build Failures
- Clear .next folder
- Clear node_modules and reinstall
- Check for circular dependencies
- Ensure all environment variables are set
Database Issues
- Check connection string
- Apply pending migrations manually (we do NOT use
prisma migrate deploy— see Database section above) - Regenerate client:
pnpm run db:generate