mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
Justin Maier
d7038c5aa8
* fix(home-blocks): hydrate the viewer own reactions on cached image blocks Home block payloads are stored in one Redis entry with no user segment AND served through edgeCacheIt with canCache left true, so every viewer is handed the same image objects with `reactions: []`. reaction.toggle acts on the database row rather than on what is drawn, so a viewer whose reaction shows un-highlighted clicks it and deletes the reaction they already had. Keeps the shared payload anonymous and hydrates on the client instead: a new reaction.getMyImageReactions procedure, and a useHydratedImageReactions hook that the three home blocks rendering ImageCard call on the list they hand to ImagesProvider, which is the same window the image detail dialog browses. The query and its grouping already existed twice inside image.service.ts; both now call one exported getUserReactionsForImages, which the new procedure is the third caller of. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(home-blocks): make the hydrated list the only one a block can render Review found the guard proved the blocks MENTION the hook, not that the hydrated array reaches the cards: keeping the call and rendering a second, un-hydrated binding restored the bug in full with the suite green. Fixed structurally rather than by a stronger string match. The hook result is now passed straight into useDedupedCappedItems, so the capped list is the only array in scope, and the guard asserts that shape. Hydrating before the cap also stops the query key churning as earlier blocks publish their dedupe claims. Also from review: - reactionQueryChunks is pure and exported, so the signed-out gate has a test. Losing it is one UNAUTHORIZED request per home block for the majority of front-page traffic. - The chunk size and the input schema's max are two copies of one number; a test pins them equal. Divergence fails zod, React Query swallows it, and the grid silently stays un-hydrated. - `reactions` is optional on the hook's item type, so the collection blocks pass their union in without a cast, and the merge reads it with `?? []`. - chunkIds moves to array-helpers; sticker.util, StickerPlacementBatchProvider and RemixGalleryBatchProvider had three copies of one body between them. - Dropped the `toContain('useHydratedImageReactions')` assertion: deleting the call leaves the import, so it barely fails. The structural assertion subsumes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(home-blocks): correct the measured cost, and stop re-asking per mount Round 2 of review. The cost model I wrote into the hook's doc comment was wrong in the direction that understates it: a FeaturedCollections block renders one section per pick and its renderCount is 5, so its picks do not share a call, and one of the three Feed blocks carries models and asks nothing. Nine requests on today's prod home page, not six. The per-request figure was measured at the post-cap shape this branch replaced; over a block's whole pre-cap pool it is 0.4-1.2 ms, about 4.6 ms of replica time for the page. Both numbers corrected rather than dropped, since a follow-up ticket quotes them. Two behaviour changes, both from the same review: - The ids are sorted before chunking, so a repeat visit can reuse the answer. Every block shuffles its pool on mount, which made the query key fresh every time and `staleTime` nearly inert. The sort lives here and NOT in `chunkIds`, whose other callers page and need insertion order. - Hydration waits on hidden preferences. `useApplyHiddenPreferences` does not block, so between the payload landing and the preference maps resolving it hands back the unfiltered pool; asking about that first spent a whole extra round of requests per cold load. The guard now accepts either nesting order. The property that kills the un-hydrated binding is the inlining, not which hook is outermost, and hydrating after the cap is a defensible shape this guard has no business forbidding - it was this branch's own design one commit ago. Its comment also stops claiming the bug is impossible to write: `filtered` is still a binding. What the guard removes is the INVITED mutation. The router-rung assertion is an allow-list of authed procedures rather than one spelling, so it reds on a loosening and not on a tightening. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(home-blocks): take the entity, not a boolean the caller derives Review found a green mutant against the subject: flipping `{ enabled: !loadingPreferences }` to `{ enabled: loadingPreferences }` switched hydration off on every render that draws a card, because the blocks return a skeleton while that flag is true. Every check in this repo stayed green - the guard reads the composition rather than the argument's value, and the pure tests prove the gate behaves correctly with the boolean it is handed, never that the boolean handed to it is right. So the boolean is gone rather than guarded. The hook takes the entity type, required and typed, and derives `entity === 'image'` in one place the existing pure tests already reach. A mistyped 'images' is now a compile error; there is no default to flip; and `type === 'image'` is no longer restated at three call sites. The `!loadingPreferences` half is deleted outright rather than moved. It was a no-op: `filterPreferences` returns `items: []` while preferences load, so the hook was already being handed an empty pool, never the unfiltered one. The comment claiming otherwise was the stated rationale for the gate, duplicated in all three blocks. The rung check reads every occurrence rather than `match`'s first. A doc comment above the declaration that quotes it - the natural thing a future editor writes - would otherwise satisfy a first-match check while the declaration underneath said something else. Three comment corrections, all of them claims that had stopped being true: - The blocks said inlining left "the only array in scope". It does not; `filtered` is still there. What inlining removes is the INVITED mistake, which is the one that happens. - The pre-cap move was justified partly on the detail dialog browsing the wider window. It does not: all three blocks hand `ImagesProvider` the capped list. The decision rests on the re-keying alone. - `chunkIds`' docstring argues against sorting and now has a caller that sorts first; it says why. The randomised shuffle control is reversed instead, so it is certainly red rather than almost surely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(home-blocks): cover the hook body, which nothing in the repo executed Two lanes independently found the same class: `reactionQueryChunks` and `mergeUserImageReactions` are well covered, and the composition between them was covered by nothing. Three one-token edits inside the hook body switch hydration off on the front page with typecheck, eslint, the guard and the whole unit suite green - the `byImageId` memo dep, the final memo's dep list, and the `query.data ?? {}` fallback. Nothing else can see those. `react-hooks/exhaustive-deps` arrives as a WARNING via next/core-web-vitals, `lint` is `eslint src/` with no `--max-warnings`, and the CI workflow says so out loud - "Errors only (no --max-warnings): the repo has 3,470 warnings". So dependency-array correctness has no automated enforcement in this repo, and one of the two dep lists sits under an `eslint-disable-next-line react-hooks/exhaustive-deps`, which makes "clean up the disabled rule" an ordinary edit for someone who has never read this ticket. The test renders the hook with a stubbed `useQueries`, asserts the un-hydrated state synchronously as a negative control, then makes the queries report data and asserts the hydrated state. It asserts a state that ARRIVES, so there is nothing on a timer to race. Two things this round got wrong first and are worth recording: - The stub originally returned its canned results whatever the hook asked for, so the non-image case passed data to a surface that had issued no query. It now returns one result per chunk, as the real `useQueries` does. The negative control is what caught it. - The first sort-direction assertion, `chunks[0][0]).toBe(1)`, could not fail: 1 sorts first lexicographically too. Measured, not assumed - the mutant stayed green on that line. It now asserts where the two orders actually disagree, at the second element. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(home-blocks): make the query stub faithful in content, not only in count Follow-up on the hook-body test, from the same lane that asked for it. The stub kept the descriptor COUNT honest and discarded what was in them, so `{ imageIds: chunk }` changed to `{ imageIds: chunks[0] }` stayed green — and in production that makes every chunk after the first ask about the first one's ids, so the back half of a large grid never hydrates and its cards go back to deleting on click. The stub now keeps the descriptors and a test asserts the hook asked each chunk about its own ids. Control: that mutation reds with `expected [ 100, 100 ] to deeply equal [ 100, 50 ]`. That path is unreachable on today's config — `FEED_FETCH_CEILING` and the collection limit are both 100, which is `REACTION_FETCH_CHUNK`, so a block's pool is always exactly one chunk. The test is there for the day one of those numbers goes up, which is a one-token edit nothing else connects to this hook. The comment says so rather than implying live coverage. Three smaller things in the same family: - The stub mapped one result per descriptor instead of slicing. A `queryResults` shorter than the descriptor list silently handed the hook a shape `useQueries` cannot produce. - `queryResults` is reset in `beforeEach`. Inheriting a previous test's value would inherit it as HYDRATED data, which is the direction that produces a false green. - `images` being hoisted out of the probe is load-bearing for the dep-list control — a fresh identity each render makes the memo recompute regardless — and nothing said so. Now it does. The `entity: 'model'` test keeps its assertion and loses its claim: the stub is what withholds the answer there, so it cannot tell a hook that filters from one that never asked. What it does prove is that the gate survives the whole body, which the pure tests next door cannot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(home-blocks): assert what came back, not only what was asked Round 6. The multi-chunk test proved the hook asked each chunk about its own ids and never looked at the hook's output, so the response half of the same shape was open: collapsing `Object.assign({}, ...queries.map(q => q.data ?? {}))` to `queries[0]?.data ?? {}` was green, and in production that drops every chunk after the first — images 101+ render un-hydrated and the first click deletes. It now asserts both ends, and the assertion on the LAST image is the only place in the suite where a chunk other than the first has to land. Same round, same root cause one argument to the right: the stub captured the descriptor's input and discarded its options, so deleting `{ staleTime: 60_000 }` was green. That is the option the round-3 sort exists to make worth having — a repeating query key buys nothing if nothing holds the answer — and the pure sort tests structurally cannot see it, because it lives in the hook. The stub now captures both arguments and the test pins it. Controls, applied and reverted: - `Object.assign(...)` -> `queries[0]?.data ?? {}`: red, `expected [] to deeply equal [ { userId: 9266475, … } ]` - `{ staleTime: 60_000 }` deleted: red - `{ imageIds: chunk }` -> `{ imageIds: chunks[0] }`: red, `expected [ 100, 100 ] to deeply equal [ 100, 50 ]` Multi-chunk remains unreachable on today's config — both ceilings are 100, which is the chunk size — and the test says so. The point of closing both halves rather than one is that the asymmetry would be invisible to whoever raises that number. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(home-blocks): cover the render production actually performs first Round 7 found the third dependency array, and unlike the multi-chunk pair this one is reachable today on every page load. `useApplyHiddenPreferences` returns `items: []` unconditionally while hidden preferences load, so every home block hands the hook an EMPTY array on its first render and the real pool on a later one. Drop the id join from the `chunks` memo's deps and it freezes at `chunkIds([], 100)`: nothing is ever asked, the merge is a permanent no-op, every card renders un-hydrated, and the first click deletes. Typecheck, eslint, the guard and all three existing hook-body cases stayed green — they all start with a NON-EMPTY pool, so a frozen `chunks` is frozen at the right value in every one of them. The new case mounts empty and populates on rerender, which is the sequence production performs. It is deliberately a separate case rather than an amendment to an existing one, and the file now says why: the fixture decides which dep list a case can see. A stable `images` binding can see the `byImageId` and final memos and cannot see `chunks`; a growing `images` reaches `chunks` and cannot see the final memo, because a fresh identity makes that one recompute regardless of its deps. The two shapes are mutually blind, so folding them together would close one and silently unarm the other. Controls, applied and reverted, all three dep lists at once to prove the new case disarmed neither of the existing ones: - `chunks` deps -> `[userId, entity]`: red on the new case - `[images, byImageId, userId]` -> `[images, userId]`: still red - `byImageId` deps -> `[queries.length]`: still red Also renamed the stub's captured parameter from `imageIds` to `input`, since the first descriptor argument is itself an object with an `imageIds` key and `d.imageIds.imageIds` read like a typo. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(home-blocks): pin the pool CHANGING, not merely growing Round 8. Keying the `chunks` memo on `images.length` instead of the id join was green against all four cases, because every one of them either holds the pool still or only grows it. In production `useApplyHiddenPreferences` hands back the PREVIOUS items during a refetch and then the new ones, and a home block's payload size comes from its config rather than its content — so a same-sized, different-membership swap is the ordinary refetch, not an exotic one. Under that mutant the hook keeps asking about the previous ids, the new images render un-hydrated, and the first click deletes. Closed by extending the existing empty-then-populated case with a same-length swap rather than adding a fifth fixture, so it disarms nothing. The fixture rule in the file's doc comment is restated to one that generalises: `chunks` has three deps and each needs something to VARY across a rerender to be visible at all, while the other two memos need something to HOLD STILL. "Stable versus growing" read as an exhaustive pair and is not one — chunk count is its own axis, and `userId` and `entity` are axes no case varies today, since `useCurrentUser` is a constant mock. Dropping either of those from the `chunks` deps is green against every case in this file. Recorded in the comment rather than quietly left out. Controls, applied and reverted, all five at once so a new case cannot silently unarm an older one: - `chunks` deps -> `[images.length, userId, entity]`: red on the extended case - `chunks` deps -> `[userId, entity]`: still red - `[images, byImageId, userId]` -> `[images, userId]`: still red - `byImageId` deps -> `[queries.length]`: still red, two cases - response merge -> `queries[0]?.data ?? {}`: still red Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(home-blocks): assert the non-image case asks for nothing, as its name says Round 9 found no green mutant. Its one free item: the non-image case asserted only the hook's OUTPUT, never that no query was issued — which the pure `it.each` next door already covers. Its name has promised the stronger thing for four rounds, and the `asked` capture that makes it possible arrived two rounds after the case was written and the case was never revisited. With the assertion it is the only place pinning that no query is issued outside `chunks`. Control: removing `entity !== 'image'` from the gate reds it with `expected [ Array(1) ] to deeply equal []`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Table of Contents
About the Project
Our goal with this project is to create a platform where people can share their stable diffusion models (textual inversions, hypernetworks, aesthetic gradients, VAEs, and any other crazy stuff people do to customize their AI generations), collaborate with others to improve them, and learn from each other's work. The platform allows users to create an account, upload their models, and browse models that have been shared by others. Users can also leave comments and feedback on each other's models to facilitate collaboration and knowledge sharing.
Tech Stack
We've built this project using a combination of modern web technologies, including Next.js for the frontend, TRPC for the API, and Prisma + Postgres for the database. By leveraging these tools, we've been able to create a scalable and maintainable platform that is both user-friendly and powerful.
- DB: Prisma + Postgres
- API: tRPC
- Front-end + Back-end: NextJS
- UI Kit: Mantine
- Storage: Cloudflare
Getting Started
To get a local copy up and running, follow these steps.
Prerequisites
- Docker, with Compose v2 (
docker compose, not the retired hyphenateddocker-compose). The database, Redis, MinIO, Meilisearch, ClickHouse and the mail catcher all run as containers. - Node.js
24.19.0. Not "20 or later" —package.jsondeclaresengines.node: ">=24.0.0 <25". The exact version lives in.nvmrc; CI installs that file's version and the production image is built on the same one, sonvm use(or any tool that reads.nvmrc) is the right way to get it. Note that nothing stops you:pnpm installonly printsWARN Unsupported engineand carries on, so the wrong major surfaces later as odd test failures rather than as a refusal at install time. - pnpm. This repo is pnpm-only, and this one is enforced —
npm installexits 1 via thepreinstallonly-allow pnpmhook.corepack enablewill pick up thepackageManagerfield for you. - Make (optional).
Installation
Standard setup
git clone https://github.com/civitai/civitai.git
cd civitai
nvm use # reads .nvmrc -> 24.19.0
corepack enable
git submodule update --init event-engine-common
cp .env-example .env.development
docker compose -f docker-compose.base.yml up -d
pnpm install
pnpm dev
Optional: Nix flake
Optional, and not the supported default. The standard setup above is what the project expects and what CI builds; nothing in the repo requires Nix, and you can ignore this section entirely. It exists because NixOS cannot use Prisma's published engines (there is no
linux-nixosbuild), so a flake is the practical way to work on this repo there. If you are not on NixOS and not already a flakes user, skip it.
The flake owns the toolchain, so you do not install Node or pnpm yourself:
git clone https://github.com/civitai/civitai.git
cd civitai
nix run .#dev
That single command checks Docker is usable, checks out the
event-engine-common submodule, creates .env.development from .env-example
if you do not already have one, starts the container stack, waits for Postgres,
runs pnpm install, and then starts the dev server on
http://localhost:3000. Every step is idempotent — it is
safe to re-run in a checkout that already works, and it will not overwrite your
.env.development or touch your data.
Useful variants:
nix run .#dev -- --no-start # bootstrap only, leave the services running
nix run .#dev -- --full # also start the signals/buzz containers (see below)
nix run .#doctor # check the flake's pins against the repo
nix flake check # the same checks, plus their own self-test
For an interactive shell with the same toolchain, use nix develop, or copy
.envrc.example to .envrc and run direnv allow to get it
automatically on cd.
With devcontainers
⚠️ Known out of step:
.devcontainer/public/docker-compose.ymlpinsmcr.microsoft.com/devcontainers/typescript-node:1-22, i.e. Node 22, which is outside this repo'sengines.noderange.pnpm installwill warn rather than stop, so the container comes up and then misbehaves in ways that look like your branch. There is no1-24tag (the template major moved on);3-24is the closest equivalent. Not changed here because it could not be exercised.
⚠️ Important Warning for Windows Users: Either clone this repo onto a WSL volume, or use the "clone repository in named container volume" command. Otherwise, you will see performance issues.
- Open the directory up in your IDE of choice
- VS Code should prompt you to "Open in container"
- If not, you may need to manually run
Dev Containers: Open Folder in Container
- If not, you may need to manually run
- For other IDEs, you may need to open the
.devcontainer/devcontainer.jsonfile, and click "Create devcontainer and mount sources" - Note: this may take some time to run initially
- VS Code should prompt you to "Open in container"
- Run
make run
The signals and buzz services
docker-compose.base.yml holds everything a contributor needs (and is also what
nix run .#dev starts). The extra services in docker-compose.yml
(signals, buzz) come from private ghcr.io images, so they only work for
internal members:
- create a GitHub personal access token with
read:packages - set it as
CR_PAT echo $CR_PAT | docker login ghcr.io -u USERNAME --password-stdin- then
docker compose up -d(or, with the flake,nix run .#dev -- --full)
After the first start
- Edit
.env.development. Most defaults work out of the box; these do not:- S3 upload credentials. Open the MinIO console at
http://localhost:9001 (username and password both
minioadmin) — note it is port 9001, port 9000 is the S3 API itself — go to "Access Keys", click "Create Access Key", and copy the key and secret intoS3_UPLOAD_KEY/S3_UPLOAD_SECRETandS3_IMAGE_UPLOAD_KEY/S3_IMAGE_UPLOAD_SECRET. WEBHOOK_TOKEN— any random string; it authenticates requests to the webhook endpoint.EMAIL_USER,EMAIL_PASS, andEMAIL_FROM(a valid email format) — any values, but they must be set for user registration to work.
- S3 upload credentials. Open the MinIO console at
http://localhost:9001 (username and password both
- On an empty database, populate it. These are slow and destructive, which is
why no bootstrap runs them for you:
make run-migrations make reseed - Visit http://localhost:3000.
Please report any issues with these commands to us on discord.
* Note that account creation will run emails through maildev, which can be accessed at http://localhost:1080.
Altering your user
- First, create an account for yourself as you normally would through the UI.
- You may wish to set yourself up as a moderator. To do so:
- Use a database editor (like DataGrip) or connect directly to the
DB (
PGPASSWORD=postgres psql -h localhost -p 15432 -U postgres civitai) - Find your user (by email or username), and change
isModeratortotrue
- Use a database editor (like DataGrip) or connect directly to the
DB (
Known limitations
Services that require external input will currently not work locally. These include:
- Orchestration (Generation, Training)
- Signals (Chat, Notifications, other real-time updates)
- Buzz
Contributing
Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the repository to your own GitHub account.
- Create a new branch for your changes.
- Make your changes to the code.
- Commit your changes and push the branch to your forked repository.
- Open a pull request on our repository.
If you would like to be more involved, consider joining the Community Development Team! For more information on the team as well as how to join, see Calling All Developers: Join Civitai's Community Development Team.
Data Migrations
Over the course of development, you may need to change the structure of the database. To do this:
- Make your changes to the
packages/civitai-db-schema/prisma/schema.full.prismafile. Notschema.prisma— that one is gitignored and regenerated fromschema.full.prismabyscripts/generate-slim-schema.json everypnpm run db:generate, so edits to it are silently overwritten. - Run
pnpm run db:migrate:empty "brief description here". This createspackages/civitai-db-schema/prisma/migrations/YYYYMMDDHHmmss_brief_description_here/migration.sqlfor you, in the one directory Prisma reads. To create it by hand instead, use that same path — not theprisma/migrationsdirectory at the repo root, which predates the monorepo layout and is no longer read. - Put your sql changes in the generated
migration.sql- These are usually simple sql commands like
ALTER TABLE ...
- These are usually simple sql commands like
- Run
make run-migrationsandmake gen-prisma - If you are adding/changing a column or table, please try to keep the
gen_seed.tsfile up to date with these changes.
Sponsors
Support this project by becoming a sponsor. Your logo will show up here with a link to your website.
License
Apache License 2.0 - Please have a look at the LICENSE for more details.