mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
docs(features): fix the six Core Systems Reference docs, which were never verified against code (#3666)
CLAUDE.md tells agents to consult docs/features/ before implementing a feature and names eight docs. Six had not been touched since0a59df66de(2026-01-15), the single commit that created them. Auditing them against HEAD found that five were wrong on the day they were written — this is not drift, the extraction was never checked. Verified at0a59df66de: the buzz enum already read Tip=0/Fee=25; src/event-engine-common never existed; nsfwBrowsingLevelsArray already contained Blocked; the getCount procedure has no commit that ever added it; the EntityMetric enums have never had Model/Download. buzz-accounts (money severity): - Documented Tip=1 and Fee=10. Actual: Tip=0 (1 is Dues), Fee=25 (10 is Training). Both fabricated numbers name a different real transaction type, so a hardcoded literal books the wrong type with no type error and no runtime failure. Replaced the restated enum with an import-and-reference-by-name instruction. - The prize-pool example was impossible: createMultiAccountBuzzTransaction has fromAccountId min(1), so it cannot pay out of the central bank, and it omitted the required externalTransactionIdPrefix (also the refund key). Split into the real multi-account spend shape and the real createBuzzTransactionMany payout shape. - Documented 3 of 9 account types; getUserBuzzAccount described as per-type balances when it returns a one-element array defaulted to yellow. notifications (rewritten): - Led with "the notification service runs in a separate repository".927e31bfee(2026-07-01) folded it into apps/notifications + packages/civitai-notifications. The main schema now holds one notification model; the other three tables are in a separate DB the monolith cannot reach. notification-cache.ts was deleted. getCount never existed (counts are trpc.user.checkNotifications). Drawer/List had their responsibilities misattributed — the behavior is in NotificationsComposed. Rewritten around the three-way ownership table. metrics-analytics (deleted, table repointed to entity-metrics.md): - entityType:'Model'/metricType:'Download' are not in the enums (only Image, and seven metric types with no Download), so the tracking snippet never compiled. - Query snippets read raw entityMetricEvents: zero files in src/ read that table; 5 read _month and 6 read entityMetricDailyAgg_v2, with a regression test pinning it. - Documented ClickHouse Buffer tables, a pattern with zero occurrences in the repo. - entity-metrics.md already covers this ground accurately and is maintained. image-resources: - Both snippets non-compiling: fetch() returns a Record keyed by image id with resources nested one level down, not a flat array. - Detection section inverted: detected=false means inherited from Post.modelVersionId, not hand-tagged. There is no per-image tagging mutation, so the doc sent readers looking for a UI that was never built. - Added the Flipt-routed replica read (the DataPacket replica lacks the ImageResourceNew backfill), which is a live correctness trap for new queries. nsfw-filtering + bitwise-flags: - Both told you to import Flags from '~/event-engine-common/utils/nsfw-utils'. ~/ maps to src/ and that path has never existed; the real class is src/shared/utils/flags.ts (69 importers). The module they pointed at has zero importers. - nsfwBrowsingLevelsFlag documented as "R, X, or XXX"; it includes Blocked (= 60). - Raw-SQL column was nsfw_level; it is quoted camelCase "nsfwLevel". - Added what the mask alone doesn't cover: onlySelectableLevels, unrated nsfwLevel=0, and the per-domain ceiling. Bit values themselves were all correct and are kept, marked as mirroring enums.ts. Line-number citations are dropped throughout — every one had rotted. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Zachary Lowden
@@ -342,7 +342,7 @@ Feature-specific documentation lives in `docs/features/`. Before implementing a
|
||||
| NSFW Filtering | [docs/features/nsfw-filtering.md](docs/features/nsfw-filtering.md) |
|
||||
| Buzz Accounts | [docs/features/buzz-accounts.md](docs/features/buzz-accounts.md) |
|
||||
| Notifications | [docs/features/notifications.md](docs/features/notifications.md) |
|
||||
| Metrics/Analytics | [docs/features/metrics-analytics.md](docs/features/metrics-analytics.md) |
|
||||
| Metrics/Analytics | [docs/features/entity-metrics.md](docs/features/entity-metrics.md) |
|
||||
| Bitwise Flags | [docs/features/bitwise-flags.md](docs/features/bitwise-flags.md) |
|
||||
| Civitai LLM Client | [docs/features/civitai-llm-client.md](docs/features/civitai-llm-client.md) |
|
||||
| Challenge Platform | [docs/features/challenge-platform.md](docs/features/challenge-platform.md) |
|
||||
|
||||
@@ -6,23 +6,28 @@ Utilities for working with bitwise flags throughout the codebase.
|
||||
|
||||
The codebase uses bitwise flags extensively for:
|
||||
- NSFW levels and content filtering
|
||||
- Permissions and access control
|
||||
- Feature flags
|
||||
- Multi-select options stored efficiently
|
||||
- Multi-select options stored efficiently (e.g. `OnboardingSteps`)
|
||||
|
||||
Feature flags are **not** bitwise — those are string-keyed and Flipt-backed, see `src/server/services/feature-flags.service.ts`.
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `event-engine-common/utils/nsfw-utils.ts` | `Flags` utility class |
|
||||
| `src/shared/constants/browsingLevel.constants.ts` | NSFW level constants |
|
||||
| `src/shared/utils/flags.ts` | `Flags` utility class |
|
||||
| `src/server/common/enums.ts` | `NsfwLevel` enum (the bit values) |
|
||||
| `src/shared/constants/browsingLevel.constants.ts` | Derived browsing-level flags + predicates |
|
||||
|
||||
## The Flags Class
|
||||
|
||||
```typescript
|
||||
import { Flags } from '~/event-engine-common/utils/nsfw-utils';
|
||||
import { Flags } from '~/shared/utils/flags';
|
||||
```
|
||||
|
||||
Only the four methods below are covered here; `flags.ts` exports about a dozen more
|
||||
(`toggleFlag`, `intersection`, `instanceToArray`, `diff`, …). Read the file rather than
|
||||
assuming this list is complete.
|
||||
|
||||
### Check if Flag is Set
|
||||
|
||||
```typescript
|
||||
@@ -88,9 +93,11 @@ const hasAny = Flags.intersects(value, checkFlags);
|
||||
|
||||
### Database Queries with Flags
|
||||
|
||||
The column is camelCase and must be quoted in raw SQL:
|
||||
|
||||
```sql
|
||||
-- Check if content matches user's browsing level
|
||||
WHERE (content.nsfw_level & :userBrowsingLevel) != 0
|
||||
WHERE (i."nsfwLevel" & :userBrowsingLevel) != 0
|
||||
|
||||
-- Check if specific flag is set
|
||||
WHERE (flags & :specificFlag) = :specificFlag
|
||||
|
||||
@@ -15,35 +15,41 @@ Buzz is Civitai's virtual currency used for:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/shared/constants/buzz.constants.ts` | Account types and transaction types |
|
||||
| `src/server/services/buzz.service.ts` | Transaction handling |
|
||||
| `packages/civitai-buzz/src/account-types.ts` | The account-type model + friendly↔API name map |
|
||||
| `src/shared/constants/buzz.constants.ts` | `TransactionType`, per-type config, derived type lists |
|
||||
| `src/server/schema/buzz.schema.ts` | Authoritative zod input contracts |
|
||||
| `src/server/services/buzz.service.ts` | Transaction handling (thin wrappers over `@civitai/buzz`) |
|
||||
| `src/server/services/bounty.service.ts` | Prize pool pattern reference |
|
||||
|
||||
## Buzz Types
|
||||
|
||||
There are different "colors" of buzz with different properties:
|
||||
The account-type model lives in `@civitai/buzz` and is re-exported through
|
||||
`buzz.constants.ts`. The three user-facing spend types:
|
||||
|
||||
```typescript
|
||||
// Spend types (user-facing)
|
||||
yellow: 'User' // NSFW-enabled, bankable, purchasable
|
||||
green: 'Green' // Bankable, purchasable
|
||||
blue: 'Generation' // Non-bankable (generation credits)
|
||||
```
|
||||
|
||||
There are **nine** account types in total, not three. Beyond the above: `red` (a spend type
|
||||
currently flagged `disabled`, so it's excluded from `buzzSpendTypes`), `creatorProgramBank`,
|
||||
`creatorProgramBankGreen`, `cashPending`, `cashSettled` and `club`. Derive lists from
|
||||
`buzzSpendTypes` / `buzzBankTypes` / `buzzPurchaseTypes` rather than hardcoding colors.
|
||||
|
||||
## Transaction Types
|
||||
|
||||
**Do not hardcode these numbers.** `TransactionType` in `src/shared/constants/buzz.constants.ts`
|
||||
is the only source of truth — always import the enum and reference members by name:
|
||||
|
||||
```typescript
|
||||
enum TransactionType {
|
||||
Tip = 1, // Tipping creators
|
||||
Reward = 5, // Prize distribution
|
||||
Purchase = 6, // Buying buzz
|
||||
Bounty = 8, // Bounty/competition fees
|
||||
BountyEntry = 9, // Entry fee collection
|
||||
Fee = 10, // Generic fees
|
||||
// ... others
|
||||
}
|
||||
import { TransactionType } from '~/shared/constants/buzz.constants';
|
||||
```
|
||||
|
||||
The values are not in any intuitive order (`Tip` is `0`, not `1`), the enum has grown over time,
|
||||
and a wrong literal books a transaction as a *different real type* with no type error and no
|
||||
runtime failure. An earlier version of this doc restated the values and got two of six wrong.
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Transaction
|
||||
@@ -59,34 +65,54 @@ await createBuzzTransaction({
|
||||
});
|
||||
```
|
||||
|
||||
### Multi-Account Transaction (Prize Pools)
|
||||
### Multi-Account Transaction (spending across buzz colors)
|
||||
|
||||
For collecting fees into a central pool and distributing prizes:
|
||||
`createMultiAccountBuzzTransaction` debits **one user** across several of their buzz colors. It is
|
||||
not the prize-pool API — `fromAccountId` is `min(1)` in the schema, so it cannot pay *out of* the
|
||||
central bank.
|
||||
|
||||
`externalTransactionIdPrefix` is required, and it is also the refund key:
|
||||
`refundMultiAccountTransaction` takes only that prefix, so a transaction created without a
|
||||
meaningful one cannot be refunded through the supported path.
|
||||
|
||||
```typescript
|
||||
import { createMultiAccountBuzzTransaction } from '~/server/services/buzz.service';
|
||||
|
||||
// Collect entry fee into central bank (account 0)
|
||||
await createMultiAccountBuzzTransaction({
|
||||
fromAccountId: userId,
|
||||
fromAccountTypes: ['yellow'], // Deduct from yellow buzz
|
||||
toAccountId: 0, // Central bank holds pool
|
||||
amount: entryFee,
|
||||
type: TransactionType.Fee,
|
||||
details: { entityId: contestId, entityType: 'Contest' },
|
||||
});
|
||||
|
||||
// Distribute prize from central bank
|
||||
await createMultiAccountBuzzTransaction({
|
||||
fromAccountId: 0, // From central bank
|
||||
fromAccountTypes: ['yellow'],
|
||||
toAccountId: winnerId,
|
||||
amount: prizeAmount,
|
||||
type: TransactionType.Reward,
|
||||
details: { entityId: contestId, entityType: 'Contest' },
|
||||
fromAccountId: ctx.user.id,
|
||||
fromAccountTypes: getAllowedAccountTypes(ctx.features),
|
||||
toAccountId: recipientUserId,
|
||||
amount: price,
|
||||
type: TransactionType.Purchase,
|
||||
description: `Early access: ${name}`,
|
||||
details: { comicChapterId: chapter.id },
|
||||
externalTransactionIdPrefix: `comic-ea-${chapter.id}-${ctx.user.id}`,
|
||||
});
|
||||
```
|
||||
|
||||
### Prize pools (paying out of the central bank)
|
||||
|
||||
Payouts from account `0` use the **single**-account API instead:
|
||||
|
||||
```typescript
|
||||
import { createBuzzTransactionMany } from '~/server/services/buzz.service';
|
||||
|
||||
await createBuzzTransactionMany(
|
||||
winners.map(({ userId }) => ({
|
||||
type: TransactionType.Reward,
|
||||
fromAccountId: 0, // central bank
|
||||
toAccountId: userId,
|
||||
toAccountType: 'blue',
|
||||
amount: prizeAmount,
|
||||
description: `Challenge Prize: ${challenge.title}`,
|
||||
externalTransactionId: `challenge-prize-${challengeId}-${userId}`,
|
||||
}))
|
||||
);
|
||||
```
|
||||
|
||||
Fee collection runs the same way in reverse, with `toAccountId: 0`. See `challenge.service.ts` for
|
||||
both directions.
|
||||
|
||||
## Central Bank (Account 0)
|
||||
|
||||
Account ID `0` is the central bank used for:
|
||||
@@ -97,8 +123,11 @@ Account ID `0` is the central bank used for:
|
||||
## Balance Checking
|
||||
|
||||
```typescript
|
||||
import { getUserBuzzAccount } from '~/server/services/buzz.service';
|
||||
import { getUserBuzzAccounts } from '~/server/services/buzz.service';
|
||||
|
||||
const account = await getUserBuzzAccount({ accountId: userId });
|
||||
// Returns balance for each buzz type
|
||||
// One entry per spend type
|
||||
const accounts = await getUserBuzzAccounts({ userId });
|
||||
```
|
||||
|
||||
`getUserBuzzAccount` (singular) returns a **one-element array defaulted to yellow** unless you pass
|
||||
`accountTypes` — it is not the per-type balance call despite the name.
|
||||
|
||||
@@ -13,10 +13,10 @@ The image resource system detects and stores information about which AI models a
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `prisma/schema.full.prisma` | `ImageResourceNew` model definition (lines 1565-1575) |
|
||||
| `src/server/services/image.service.ts` | `getImageResources()`, `getImageResourcesFromImageId()` |
|
||||
| `src/server/redis/caches.ts` | `imageResourcesCache` (lines 976-1018) |
|
||||
| `prisma/programmability/get_image_resources.sql` | Detection function |
|
||||
| `packages/civitai-db-schema/prisma/schema.full.prisma` | `ImageResourceNew` model definition |
|
||||
| `src/server/services/image.service.ts` | `getImageResourcesFromImageId()`, `createImageResources()` |
|
||||
| `src/server/redis/caches.ts` | `imageResourcesCache`, `ImageResourceCacheItem` |
|
||||
| `packages/civitai-db-schema/prisma/programmability/get_image_resources.sql` | Detection function |
|
||||
|
||||
## Schema
|
||||
|
||||
@@ -34,33 +34,57 @@ model ImageResourceNew {
|
||||
|
||||
### Fetching Resources for an Image
|
||||
|
||||
`fetch()` takes an array and returns a `Record` keyed by image id, and the resources are nested
|
||||
one level down — so it's always a two-step unwrap:
|
||||
|
||||
```typescript
|
||||
import { imageResourcesCache } from '~/server/redis/caches';
|
||||
|
||||
// Get all resources used in an image (cached)
|
||||
const resources = await imageResourcesCache.fetch(imageId);
|
||||
|
||||
// Resources include:
|
||||
// - modelVersionId: The model version used
|
||||
// - strength: Weight/strength if applicable (for LoRAs)
|
||||
// - detected: Whether this was auto-detected vs user-specified
|
||||
const byImage = await imageResourcesCache.fetch([imageId]);
|
||||
const resources = byImage[imageId]?.resources ?? [];
|
||||
```
|
||||
|
||||
Each entry is an `ImageResourceCacheItem` (see `caches.ts` for the authoritative shape). It carries
|
||||
more than the three columns of the underlying table — `modelId`, `modelName`, `modelType`,
|
||||
`versionName`, `baseModel`, `poi` and `minor` are already denormalized in, so needing any of those
|
||||
is **not** a reason to hit the database again.
|
||||
|
||||
### Validating Resource Usage
|
||||
|
||||
```typescript
|
||||
const resources = await imageResourcesCache.fetch(imageId);
|
||||
const usedModelVersionIds = resources.map(r => r.modelVersionId);
|
||||
const byImage = await imageResourcesCache.fetch([imageId]);
|
||||
const usedModelVersionIds = (byImage[imageId]?.resources ?? []).map((r) => r.modelVersionId);
|
||||
|
||||
// Check if image only used allowed resources
|
||||
const allowedResources = [123, 456, 789]; // model version IDs
|
||||
const isValid = usedModelVersionIds.every(id => allowedResources.includes(id));
|
||||
const isValid = usedModelVersionIds.every((id) => allowedResources.includes(id));
|
||||
```
|
||||
|
||||
Note that the shipped challenge/contest validation does **not** go through the cache — it joins
|
||||
`ImageResourceNew` directly (`challenge.service.ts`). Use the cache for display paths; check the
|
||||
existing call site before adding a new validation path.
|
||||
|
||||
### Reads are flag-routed away from the replica
|
||||
|
||||
`imageResourcesCache`'s lookup consults the `IMAGE_RESOURCE_USE_WRITE` Flipt flag and otherwise
|
||||
routes through `getDbWithoutLagBatch`, because the DataPacket replica is missing the
|
||||
`ImageResourceNew` backfill. The same routing is open-coded at the other `ImageResourceNew` call
|
||||
sites. **A new query written against the plain read replica will silently return no rows there** —
|
||||
copy the routing from an existing call site rather than reaching for `dbRead`.
|
||||
|
||||
## Detection
|
||||
|
||||
Resources can be:
|
||||
- **Auto-detected**: Extracted from image metadata (generation parameters)
|
||||
- **User-specified**: Manually tagged by the uploader
|
||||
The `detected` flag records *how the resource was associated with the image*, not who did it:
|
||||
|
||||
The `detected` field distinguishes between these cases.
|
||||
- **`detected: true`** — extracted from the image's generation metadata. Four of the five branches
|
||||
in `get_image_resources.sql` produce this.
|
||||
- **`detected: false`** — inherited from the post's linked model version (`Post.modelVersionId`),
|
||||
via the one remaining branch.
|
||||
|
||||
There is no per-image "tag a resource" mutation, so `detected: false` never means hand-tagged by a
|
||||
user. The only production writer is `createImageResources()`, fed entirely by the SQL function.
|
||||
|
||||
Note `strength` is stored as `round(weight * 100)` — a weight of `1.0` is `100`, not `1`.
|
||||
|
||||
A second TypeScript reimplementation of the detection logic now lives in
|
||||
`generation.service.ts`, whose comments state it mirrors the SQL's stages. If you change the
|
||||
detection rules, change both.
|
||||
|
||||
@@ -1,139 +0,0 @@
|
||||
# Metrics & Analytics
|
||||
|
||||
Track events and metrics using ClickHouse for analytics and reporting.
|
||||
|
||||
## Overview
|
||||
|
||||
The metrics system uses ClickHouse for high-volume event tracking and analytics. It supports:
|
||||
- Entity-level metrics (views, downloads, likes, etc.)
|
||||
- User activity tracking
|
||||
- Custom event tracking
|
||||
- Real-time aggregations
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/server/clickhouse/client.ts` | ClickHouse client and `Tracker` class |
|
||||
| `src/server/utils/metric-helpers.ts` | Helper functions |
|
||||
| `src/server/metrics/` | Metric processors by entity type |
|
||||
| `src/server/metrics/base.metrics.ts` | Metric processor factory |
|
||||
|
||||
## Entity Metrics
|
||||
|
||||
### Tracking Events
|
||||
|
||||
```typescript
|
||||
// In a tRPC procedure or service
|
||||
await ctx.track.entityMetric({
|
||||
entityType: 'Model',
|
||||
entityId: modelId,
|
||||
metricType: 'Download',
|
||||
metricValue: 1,
|
||||
});
|
||||
|
||||
// Common metric types:
|
||||
// - View, Download, Like, Dislike, Comment, Share
|
||||
```
|
||||
|
||||
### ClickHouse Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE entityMetricEvents (
|
||||
entityType LowCardinality(String),
|
||||
entityId Int32,
|
||||
userId Int32,
|
||||
metricType LowCardinality(String),
|
||||
metricValue Int32,
|
||||
createdAt DateTime64(3)
|
||||
) ENGINE = MergeTree()
|
||||
ORDER BY (entityType, entityId, createdAt);
|
||||
```
|
||||
|
||||
## Custom Event Tables
|
||||
|
||||
For high-volume or specialized tracking, create dedicated tables:
|
||||
|
||||
```sql
|
||||
CREATE TABLE my_feature_events (
|
||||
feature_id UInt32,
|
||||
user_id UInt32,
|
||||
action LowCardinality(String),
|
||||
metadata String, -- JSON for flexible data
|
||||
created_at DateTime DEFAULT now()
|
||||
) ENGINE = MergeTree()
|
||||
PARTITION BY toYYYYMM(created_at)
|
||||
ORDER BY (feature_id, created_at)
|
||||
TTL created_at + INTERVAL 90 DAY; -- Auto-cleanup old data
|
||||
```
|
||||
|
||||
## Buffer Tables
|
||||
|
||||
ClickHouse prefers batch inserts. Use buffer tables for high-frequency writes:
|
||||
|
||||
```sql
|
||||
-- Main table
|
||||
CREATE TABLE my_events (...) ENGINE = MergeTree() ...;
|
||||
|
||||
-- Buffer that auto-flushes to main table
|
||||
CREATE TABLE my_events_buffer AS my_events
|
||||
ENGINE = Buffer(
|
||||
default, -- database
|
||||
my_events, -- destination table
|
||||
16, -- num_layers
|
||||
10, 100, -- min/max seconds
|
||||
10000, 1000000, -- min/max rows
|
||||
10000000, 100000000 -- min/max bytes
|
||||
);
|
||||
|
||||
-- Write to buffer, reads from main table
|
||||
INSERT INTO my_events_buffer VALUES (...);
|
||||
SELECT * FROM my_events; -- Includes buffered data
|
||||
```
|
||||
|
||||
## Querying Metrics
|
||||
|
||||
### Aggregations
|
||||
|
||||
```typescript
|
||||
import { clickhouse } from '~/server/clickhouse/client';
|
||||
|
||||
const result = await clickhouse.query({
|
||||
query: `
|
||||
SELECT
|
||||
entityId,
|
||||
countIf(metricType = 'View') as views,
|
||||
countIf(metricType = 'Download') as downloads
|
||||
FROM entityMetricEvents
|
||||
WHERE entityType = 'Model'
|
||||
AND createdAt > now() - INTERVAL 7 DAY
|
||||
GROUP BY entityId
|
||||
ORDER BY views DESC
|
||||
LIMIT 100
|
||||
`,
|
||||
});
|
||||
```
|
||||
|
||||
### Time Series
|
||||
|
||||
```typescript
|
||||
const dailyStats = await clickhouse.query({
|
||||
query: `
|
||||
SELECT
|
||||
toDate(createdAt) as date,
|
||||
count() as events
|
||||
FROM entityMetricEvents
|
||||
WHERE entityType = 'Model' AND entityId = {modelId:UInt32}
|
||||
GROUP BY date
|
||||
ORDER BY date
|
||||
`,
|
||||
params: { modelId },
|
||||
});
|
||||
```
|
||||
|
||||
## Metric Event Watcher
|
||||
|
||||
For database-triggered metrics (CDC pattern), see the `metric-event-watcher` service which uses Debezium to:
|
||||
1. Watch PostgreSQL table changes
|
||||
2. Process events and update ClickHouse
|
||||
3. Maintain materialized views for aggregations
|
||||
@@ -1,133 +1,113 @@
|
||||
# Notifications System
|
||||
|
||||
This document explains how the notification system works in Civitai. The notification service itself runs in a separate repository, but this codebase handles the client-side implementation, notification processing, and user interaction.
|
||||
How notifications are produced, fanned out, read and displayed.
|
||||
|
||||
## Key Files
|
||||
## Where things live
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/server/notifications/` | Notification processors by feature |
|
||||
| `src/server/notifications/base.notifications.ts` | `createNotificationProcessor()` factory |
|
||||
| `src/server/notifications/utils.notifications.ts` | Processor registry |
|
||||
| `src/server/services/notification.service.ts` | `createNotification()` service |
|
||||
| `src/server/jobs/send-notifications.ts` | Background processing job |
|
||||
| `src/components/Notifications/` | Client-side notification components |
|
||||
The notification domain was moved out of the monolith (and out of the old external
|
||||
`notification-server` repo) into an in-repo app plus a shared package. Three places now split the
|
||||
work, and "which of the three owns this?" is the first question to answer for any change:
|
||||
|
||||
## Architecture Overview
|
||||
| Location | Owns |
|
||||
|---|---|
|
||||
| `apps/notifications/` | The notification database (sole owner), the settings opt-out filter on create, the fan-out poll worker, the read/count/mark queries, the per-user unread cache, and signal emission |
|
||||
| `packages/civitai-notifications/` | The zod contracts, `NotificationCategory` and the signal constants, and the HTTP client — the single source of truth shared by every producer |
|
||||
| `src/server/notifications/` | Only the per-feature **processors** (`prepareQuery`/`prepareMessage`), the `detail-fetchers/` main-DB enrichment, and the configured client instance in `client.ts` |
|
||||
|
||||
The notification system consists of several key components:
|
||||
The external `notification-server` repo is retired.
|
||||
|
||||
- **Notification Processors**: Define different types of notifications and their behavior
|
||||
- **Notification Service**: Handles creation, retrieval, and management of notifications
|
||||
- **Client Components**: UI components for displaying and interacting with notifications
|
||||
- **Real-time Updates**: WebSocket-based real-time notification delivery via Signals
|
||||
- **Caching Layer**: Redis-based caching for notification counts and data
|
||||
## Database
|
||||
|
||||
## Database Schema
|
||||
**The monolith cannot reach the notification database.** Only `UserNotificationSettings` is in the
|
||||
main Prisma schema. `Notification`, `UserNotification` and `PendingNotification` live in a
|
||||
physically separate database reached exclusively by `apps/notifications`, over its own pool
|
||||
(`NOTIFICATION_DB_URL`). Those env vars are optional in the monolith precisely because it no longer
|
||||
connects.
|
||||
|
||||
### Core Tables
|
||||
- `Notification`: Stores notification content and metadata
|
||||
- `UserNotification`: Links notifications to users with read/unread status
|
||||
- `PendingNotification`: Queue for notifications to be processed
|
||||
- `UserNotificationSettings`: User preferences for notification types
|
||||
If you need notification rows, add an endpoint to `apps/notifications` — do not add a Prisma model.
|
||||
|
||||
### Key Fields
|
||||
- `type`: Notification type (e.g., 'model-download-milestone', 'comment-created')
|
||||
- `category`: Notification category for grouping (Comment, Update, Milestone, etc.)
|
||||
- `details`: JSON object containing notification-specific data
|
||||
- `key`: Unique identifier for deduplication
|
||||
### Key fields
|
||||
|
||||
## Notification Categories
|
||||
- `type` — notification type (e.g. `model-download-milestone`)
|
||||
- `category` — grouping, see below
|
||||
- `details` — JSON payload
|
||||
- `key` — dedupe identifier
|
||||
- `dedupeKey` / `debounceSeconds` — first-class since the extraction; a cross-schema refine forbids
|
||||
combining them
|
||||
|
||||
Located in `src/server/common/enums.ts`:
|
||||
## Categories
|
||||
|
||||
```typescript
|
||||
enum NotificationCategory {
|
||||
Comment = 'Comment',
|
||||
Update = 'Update',
|
||||
Milestone = 'Milestone',
|
||||
Bounty = 'Bounty',
|
||||
Buzz = 'Buzz',
|
||||
Creator = 'Creator',
|
||||
System = 'System',
|
||||
Other = 'Other',
|
||||
}
|
||||
```
|
||||
Defined once in `packages/civitai-notifications/src/constants.ts` and re-exported from
|
||||
`src/server/common/enums.ts`. It is a `const` tuple plus a derived union, **not** a TypeScript
|
||||
`enum`, so import it rather than restating the members — the list has grown before and will again.
|
||||
|
||||
## Notification Types & Processors
|
||||
## Processors
|
||||
|
||||
Notification processors are defined in `src/server/notifications/` and handle:
|
||||
- Query preparation for finding relevant events
|
||||
- Message formatting for display
|
||||
- Category assignment and settings
|
||||
Processors live in `src/server/notifications/*.notifications.ts`, are built with
|
||||
`createNotificationProcessor()` from `base.notifications.ts`, and are collected in the
|
||||
`utils.notifications.ts` registry. Each defines query preparation, message formatting, and its
|
||||
category/settings.
|
||||
|
||||
### Key Processor Files
|
||||
- `model.notifications.ts` - Model-related notifications (downloads, likes, milestones)
|
||||
- `comment.notifications.ts` - Comment notifications
|
||||
- `reaction.notifications.ts` - Like/reaction notifications
|
||||
- `follow.notifications.ts` - User follow notifications
|
||||
- `bounty.notifications.ts` - Bounty-related notifications
|
||||
- `buzz.notifications.ts` - Buzz transaction notifications
|
||||
- And many more...
|
||||
The registry is large and covers far more than the obvious feature areas — read it rather than
|
||||
assuming a notification type doesn't exist yet.
|
||||
|
||||
Important note: I've marked where these can moved as "Moveable" creation as opposed to job-based. The only notifications that should really be handled in jobs are ones that are time-based, such as milestones or hourly/daily aggregations. The rest should eventually be moved to on-demand creation.
|
||||
Processors marked **"Moveable"** are ones that could become on-demand creation instead of job-based.
|
||||
Only genuinely time-based notifications — milestones, hourly/daily aggregations — need to stay in
|
||||
the `send-notifications` job.
|
||||
|
||||
Notifications can be manually created via `createNotification()` in `notification.service.ts`:
|
||||
## Creating a notification
|
||||
|
||||
## Client-Side Implementation
|
||||
`createNotification()` in `src/server/services/notification.service.ts` is still the entry point,
|
||||
but its contract changed with the extraction: it is now an **HTTP call to `apps/notifications` that
|
||||
never throws on transport failure**. Client errors are swallowed and logged centrally.
|
||||
|
||||
### Key Components
|
||||
That means a caller cannot treat a successful return as proof the notification was stored, and
|
||||
must not put it inside a transaction expecting rollback semantics. The same applies to
|
||||
`markNotificationsRead`.
|
||||
|
||||
#### NotificationBell (`src/components/Notifications/NotificationBell.tsx`)
|
||||
- Shows notification count indicator
|
||||
- Opens notification drawer on click
|
||||
- Hides on notification pages
|
||||
## Client
|
||||
|
||||
#### NotificationDrawer (`src/components/Notifications/NotificationsDrawer.tsx`)
|
||||
- Displays notification list in a drawer
|
||||
- Handles infinite scrolling
|
||||
- Category filtering
|
||||
| Component | Responsibility |
|
||||
|---|---|
|
||||
| `NotificationBell.tsx` | Unread indicator; opens the drawer; hides on notification pages |
|
||||
| `NotificationsDrawer.tsx` | Mantine `Drawer` shell only — delegates to `NotificationsComposed` |
|
||||
| `NotificationsComposed.tsx` | The real behavior: infinite scroll, category filtering, mark-as-read |
|
||||
| `NotificationList.tsx` | Presentational render only; takes an `onItemClick` prop |
|
||||
|
||||
#### NotificationList (`src/components/Notifications/NotificationList.tsx`)
|
||||
- Renders individual notifications
|
||||
- Mark as read functionality
|
||||
- Pagination support
|
||||
Changes to list behavior almost always belong in `NotificationsComposed.tsx`, not in the drawer or
|
||||
the list.
|
||||
|
||||
## User Settings
|
||||
Realtime delivery is via Signals — the fan-out worker POSTs per affected user, and
|
||||
`notifications.utils.ts` subscribes.
|
||||
|
||||
Users can control notification preferences in `src/components/Account/NotificationsCard.tsx`:
|
||||
## User settings
|
||||
|
||||
- Toggle specific notification types on/off
|
||||
- Settings stored in `UserNotificationSettings` table
|
||||
- Respected during notification creation
|
||||
`src/components/Account/NotificationsCard.tsx`. Settings are stored in `UserNotificationSettings`
|
||||
(the one table still in the main schema) and applied as an opt-out filter inside
|
||||
`apps/notifications` on create.
|
||||
|
||||
## Caching Strategy
|
||||
## Caching
|
||||
|
||||
Notification counts are cached in Redis via `notification-cache.ts`:
|
||||
Unread counts are cached per user in Redis, keyed by category, by
|
||||
`apps/notifications/src/lib/server/cache.ts`. The monolith's old `notification-cache.ts` was deleted
|
||||
in the extraction.
|
||||
|
||||
- User-level caching of unread counts by category
|
||||
- Cache invalidation on read/creation
|
||||
- Optimistic updates for UI responsiveness
|
||||
## API
|
||||
|
||||
## API Endpoints
|
||||
tRPC routes in `src/server/routers/notification.router.ts` — three procedures, all scope-gated:
|
||||
|
||||
### tRPC Routes (`src/server/routers/notification.router.ts`)
|
||||
- `getAllByUser` - Get paginated user notifications
|
||||
- `markRead` - Mark notifications as read
|
||||
- `getCount` - Get notification counts (cached)
|
||||
- `getAllByUser` — paginated notifications
|
||||
- `markRead` — mark as read
|
||||
- `updateUserSettings` — notification preferences
|
||||
|
||||
### Best Practices
|
||||
|
||||
- **Deduplication**: Use unique keys to prevent duplicate notifications
|
||||
- **User Preferences**: Respect user notification settings
|
||||
- **Performance**: Optimize queries for large datasets
|
||||
- **Real-time**: Emit signals for immediate UI updates
|
||||
- **Categories**: Group related notifications logically
|
||||
**Counts do not come from this router.** The unread count is `trpc.user.checkNotifications`.
|
||||
|
||||
## Debugging
|
||||
|
||||
### Common Issues
|
||||
- **Missing Notifications**: Check user settings and deduplication logic
|
||||
- **Duplicate Notifications**: Verify unique key generation
|
||||
- **Performance**: Monitor notification job execution times
|
||||
- **Cache Issues**: Clear Redis cache if counts seem incorrect
|
||||
- **Missing notifications** — check the user's settings opt-out and the dedupe key. Remember create
|
||||
is best-effort, so a silent failure is logged, not thrown.
|
||||
- **Duplicates** — verify `key` generation, and whether `dedupeKey`/`debounceSeconds` are being
|
||||
combined.
|
||||
- **Counts wrong** — the cache is in `apps/notifications`, not this codebase.
|
||||
- **Nothing fanning out** — the worker is env-gated and defaults off; confirm it's enabled in the
|
||||
environment you're looking at before assuming a code bug.
|
||||
|
||||
@@ -10,13 +10,14 @@ The NSFW filtering system uses bitwise flags to efficiently filter content based
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/server/common/enums.ts` | `NsfwLevel` enum (lines 277+) |
|
||||
| `src/shared/constants/browsingLevel.constants.ts` | Level constants and utilities |
|
||||
| `event-engine-common/utils/nsfw-utils.ts` | `Flags` utility class |
|
||||
| `src/server/common/enums.ts` | `NsfwLevel` enum — source of truth for the bit values |
|
||||
| `src/shared/constants/browsingLevel.constants.ts` | Derived flags, ceilings and predicates |
|
||||
| `src/shared/utils/flags.ts` | `Flags` utility class |
|
||||
| `src/server/services/image.service.ts` | The feed filter in practice |
|
||||
|
||||
## NsfwLevel Enum
|
||||
|
||||
The levels are bitwise flags, allowing content to be tagged with multiple levels and users to allow multiple levels:
|
||||
The levels are bitwise flags, allowing content to be tagged with multiple levels and users to allow multiple levels. Mirrored here because they're the whole subject of this doc — `src/server/common/enums.ts` is authoritative:
|
||||
|
||||
```typescript
|
||||
enum NsfwLevel {
|
||||
@@ -34,19 +35,21 @@ enum NsfwLevel {
|
||||
### Checking Content Visibility
|
||||
|
||||
```typescript
|
||||
import { Flags } from '~/event-engine-common/utils/nsfw-utils';
|
||||
import { Flags } from '~/shared/utils/flags';
|
||||
|
||||
// Check if content is visible to user
|
||||
const isVisible = Flags.intersects(contentNsfwLevel, userBrowsingLevel);
|
||||
// Returns true if ANY bits overlap (content allowed for user)
|
||||
```
|
||||
|
||||
This is the core predicate but **not** the whole filter — see "What the mask alone doesn't tell you" below.
|
||||
|
||||
### Checking if Content is NSFW
|
||||
|
||||
```typescript
|
||||
import { nsfwBrowsingLevelsFlag } from '~/shared/constants/browsingLevel.constants';
|
||||
|
||||
// Check if content is NSFW (R, X, or XXX)
|
||||
// R | X | XXX | Blocked === 60. Note it includes Blocked, not just the three mature levels.
|
||||
const isNsfw = Flags.intersects(level, nsfwBrowsingLevelsFlag);
|
||||
```
|
||||
|
||||
@@ -67,14 +70,10 @@ Flags.intersects(contentLevel, browsingLevel); // true
|
||||
|
||||
### Filtering Database Queries
|
||||
|
||||
Prisma has no bitwise operator, so this is always raw SQL. The column is camelCase and must be quoted:
|
||||
|
||||
```typescript
|
||||
// In Prisma queries, use bitwise AND
|
||||
where: {
|
||||
nsfwLevel: {
|
||||
// Content level AND user level != 0
|
||||
// This is typically done with raw SQL or computed fields
|
||||
}
|
||||
}
|
||||
Prisma.sql`AND (i."nsfwLevel" & ${browsingLevel}) != 0`
|
||||
```
|
||||
|
||||
### Setting Content Levels
|
||||
@@ -85,6 +84,14 @@ When creating content that accepts multiple NSFW levels:
|
||||
const allowedLevels = NsfwLevel.PG | NsfwLevel.PG13 | NsfwLevel.R; // = 7
|
||||
```
|
||||
|
||||
## What the mask alone doesn't tell you
|
||||
|
||||
`Flags.intersects(contentLevel, browsingLevel)` is necessary but not sufficient. The real feed path in `image.service.ts` layers three more rules on top, and code that reimplements only the mask will not match it:
|
||||
|
||||
- **`Blocked` is stripped before the comparison.** Browsing levels go through `onlySelectableLevels()` (`browsingLevel.constants.ts`) first, so a user's stored level never admits `Blocked` content.
|
||||
- **Unrated content (`nsfwLevel = 0`) matches nothing** — `0 & anything === 0` — so it needs its own branch. The public feed excludes it explicitly; owners see their own, and moderators get it added only when their level already intersects NSFW.
|
||||
- **A per-domain ceiling clamps the result.** `domainBrowsingCeiling` limits blue/green domains to SFW regardless of user preference; red is unclamped.
|
||||
|
||||
## See Also
|
||||
|
||||
- [Bitwise Flag Utilities](./bitwise-flags.md) - General flag manipulation utilities
|
||||
|
||||
Reference in New Issue
Block a user