mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
main
175 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Zachary Lowden
|
d872ab698b |
feat(apps): migrate /apps/installed to /apps/activity — widen past the slot flag, default to the feed, Build to the end of the sub-nav (#4699)
* feat(apps): migrate /apps/installed to /apps/activity — widen past the slot flag, default to the feed, move Build to the end of the sub-nav
The page stopped being an installs surface. It now opens on the activity feed,
its Installs tab is gated on the model-slot flag alone, and the page itself is
reachable by anyone holding either runtime flag. The route name follows.
1. ROUTE. src/pages/apps/installed.tsx moves to src/pages/apps/activity.tsx and
/apps/installed gains a 301 in next.config.mjs, on the precedent of the
/apps/build consolidation (statusCode: 301, not permanent: true — Next maps
permanent to 308). The page component is MOVED, not stubbed: a stub whose
only job is to redirect reads as a live route, and the appsPageWidths fs-walk
would demand a width classification for a page that never renders. Every
in-repo link is repointed; the prose is swept.
2. GATE. getServerSideProps now calls the shared resolveActivityPageAccess,
which calls the shared canAccessAppsActivity = appBlocks || appBlocksPages.
appBlocks is the model-SLOT flag; someone who has only ever run a full-page
app (/apps/run/<slug>, gated on appBlocksPages) has generations, scope-gated
API calls and Buzz spends recorded against them and no slot install at all.
Refusing them a page called "Activity" was the dishonest half of the rename.
The page body re-checks with the same predicate, so the two cannot drift.
3. INSTALLS TAB. Narrows to features.appBlocks — the tab's content is slot
subscriptions and per-model installs, which is exactly what that flag
governs. This is only NON-VACUOUS because of (2): with the page gate
unchanged the tab check could never be false. The PANEL is gated with the
tab, since a Tabs.Panel with no tab still mounts its children.
4. TABS ARE URL-BACKED. Controlled value + onChange writing ?tab= through
router.replace(..., { shallow: true }); the default tab drops the key rather
than writing ?tab=activity. Resolution rules live in a pure module
(appsActivityTabs.ts) so they run in the blocking node tier: an unknown,
repeated, or prototype-key value falls back, and so does ?tab=subscriptions
for a viewer without the slot flag — handing that to Tabs.value would render
a bar with nothing active over an empty panel.
5. SUB-NAV ORDER. Marketplace → Activity → Invites → Revenue → Build → Review.
Build was first on the argument that it is "the front door for someone who
has not built anything yet"; it is the narrowest-audience row in the table
(canAccessAppsBuild), so leading with it put the smallest cohort's
destination in the position that reads as what the bar is for. The stale
comment is rewritten rather than moved. The chrome's two /apps/installed
items are repointed and the platform-nav one relabelled to "App activity".
6. HEADER CTA. The AppsPageLayout "Browse marketplace" button is gone —
Marketplace is a sub-nav tab. The three in-empty-state anchors stay: an empty
panel with no way forward is a dead end.
7. WHEN COLUMN. Relative, via the existing DaysFromNow, with the absolute stamp
still in the tooltip (and now also in the <time datetime> attribute).
8. APP NAME LINKS TO THE STORE DETAIL, GATED. getListingDetailHref(slug), gated
on hasAppsStoreAccess read through useOptionalFeatureFlags so an absent
provider fails CLOSED — AppNameCrumb's solution copied, not re-derived.
/apps/store-preview/<slug> getServerSideProps-gates on that predicate and
answers notFound, and appListings.getAppDetail throws NOT_FOUND at scope
none, so an ungated link is a 404 affordance: the class #4668 shipped and
#4685 exists to prevent.
9. COLUMN LEDGER RE-MEASURED. The appsWideLayout EXEMPT entry justified itself
with a measurement taken while When rendered a fixed-width YYYY-MM-DD HH:mm
stamp, so it no longer described the table. Redone at 768/1200/1440/2560 on
the same fixture: When gave up ~10% of its width at every viewport (411.09 to
372.33 at 2560) and the other four columns absorbed it; ROW HEIGHT is 36.19
at all four widths, before and after. The no-surplus argument is a claim
about max-content sum versus container width at 768, and a narrower When
makes that sum smaller — the exemption is unchanged. The before@2560 row
reproduces the figure already recorded in that comment, which is the control
on the re-measurement.
TESTS. Seven new files. resolveActivityPageAccess.test.ts and
appsActivityTabs.test.ts run in the blocking node tier; apps-activity-redirect
invokes next.config.mjs's redirects() rather than scanning the file, and carries
the same positive controls as its /apps/build sibling. Two browser files mount
the real panel and the real page. Every new test was measured RED at
origin/main: 8/8 in AppActivityPanel.storeGate, 9/11 in AppActivityPage (the two
that pass there are the positive-direction cases main already satisfies).
The criterion-8 gate is mutation-checked: deleting "!canSeeStore ||" from
ActivityAppName makes exactly the two gate tests fail, on their own assertion —
"an ineligible viewer must not be given an anchor: expected 'A' not to be 'A'" —
while the three eligible-viewer tests stay green, which is what makes the red
attributable to the gate rather than to the feature.
Also re-cut the blanking-ratio bound in appsStoreAccessCallSites from 0.55 to
0.65. Measured on origin/main it sat at 0.5486 against a 0.55 bound — 0.0014 of
headroom, so any added documentation tripped it — and it now carries a negative
control on the real file (the historical apostrophe desync) proving the bound
can still fire.
* fix(apps): repoint the two label-only references the route sweep could not see
Both were found by the FULL component tier, not by the sweep, and that is the
point worth recording: the sweep keyed on the route string "/apps/installed",
so a reference that names only the LABEL was structurally invisible to it.
Sampling one surface is not enumerating the surface.
- AppBlockChromeMobileShell.browser.test.tsx asserts the platform nav folds into
the phone-width overflow sheet by looking for its label for this route. That
label went "Installed apps" to "App activity" with the rename; the test's own
RED-at-origin note is updated to say the label moved rather than the claim.
- subNavRowsMatchPageGates.test.ts named the summary-driven rows in prose as
"Installed / Invites / Revenue / Review".
* fix(apps): give the Activity tab to the page-app cohort, and stop offering links into a notFound
Audit follow-ups on #4699. The page gate widened to `appBlocks || appBlocksPages`,
but the sub-nav row pointing at it did not — and two affordances inside it now
lead somewhere the newly-admitted cohort cannot go.
1. `blocks.getNavSummary` gains `hasActivity` — true when the viewer has a row in
EITHER table the activity feed actually walks (`block_buzz_attribution` as the
SPENDER, `block_scope_invocations`). The scope probe mirrors the feed's own
WHERE clause, external-OAuth rows excluded, so the flag cannot light a tab over
an empty page. Two bounded `findFirst({ select: { id } })` in the existing
Promise.all; the flag-off shape gains the key too.
2. The Activity row keys on `s.hasInstalls || s.hasActivity`. `hasInstalls` is a
`block_user_subscriptions` row — a SLOT install — and a full-page app is
stateless by design, so a viewer who only runs page apps had activity and no
tab. Both terms are documented at the row so neither gets "simplified" away.
3. The two empty-state `/apps` anchors are gated on `hasAppsStoreAccess`.
`appBlocksPages` is not one of that predicate's disjuncts, so for the cohort
this PR admits they were links into a `notFound`. The CTA is omitted rather
than reworded; `pages/apps/activity.tsx` joins the store-gate call-site ledger.
4. `appSlug` is a listing slug or NULL — never an `AppBlock` primary key. Both
feeds emitted `r.appBlock?.blockId ?? r.appBlockId`, whose fallback is the FK,
so an unresolved join produced `/apps/store-preview/<pk>`: a guaranteed 404 on
exactly the rows where the app is least resolvable. Fixed at the source, not
with a per-row fetch; `ActivityAppName` renders plain text when the slug is
absent, and the slug badge is dropped with it.
5. `subNavRowsMatchPageGates` pins the Activity row against `/apps/activity`'s own
resolver, and its header claim that summary-driven rows have "no second copy to
disagree with" is corrected — WHICH fields a row ORs together is exactly such a
copy.
6. Two stale comment claims fixed: `AppsSubNav`'s `getNavSummary` `enabled`
rationale ("all point at pages that themselves 404 without appBlocks" — Activity
no longer does), and the test header above.
7. The `When` cell drops its Mantine `Tooltip`: `DaysFromNow` already renders a
`<time title>`, so one hover raised two tooltips formatting the same instant two
different ways. `live` is now passed so a long-lived tab does not freeze at
"20m ago".
8. The run-frame permissions drawer passes `linkable={false}`: the shared panel is
mounted OVER a running full-page app there, and the new app-name link would
top-level-navigate the user out of it. Recorded on both components.
Red→green measured per change; the two gates are mutation-checked (details in the
PR body). The double-tooltip guard was rewritten after its first, structural form
survived the mutation — a closed Mantine Tooltip is invisible in the DOM.
* fix(apps): single-source the scope-activity predicate, and correct a residual stated wider than it is
Audit round 2. Two findings actionable; the headline one was DEMONSTRATED rather
than argued, and this fixes it the same way.
F2 (guard) — "the probe MIRRORS the feed" was TWO copies of one literal, and
nothing pinned the relationship. `getNavSummary`'s `hasActivity` probe and
`listMyScopeInvocations` each asserted their OWN `where` against a hand-written
literal in their own suite; neither read the other. The audit tightened the
FEED's clause and updated the FEED's own literal — exactly the edit a developer
making that change would make — and BOTH suites stayed green at 65/65 while the
probe silently over-matched. That is the "docstring names a RELATIONSHIP, body
inspects one SIDE" shape.
Fixed by construction, not by assertion: `GLOBAL_SCOPE_ACTIVITY_OR` is exported
once from `user-app-surface.service.ts`; the feed spreads it and
`blocks.router.ts` imports it. A future edit moves both.
Proof, same mutation: tightening the predicate at its single source now reds
THREE tests across ALL THREE files — the feed's, the probe's, and the new
`scopeActivityPredicate.test.ts` — where it previously reddened none.
The new seam test deliberately does NOT re-spell the clause as a third
literal; it asserts the SHAPE and that neither call site carries its own copy.
F5 (comment) — the documented residual said a viewer holding `appBlocksPages`
without `appBlocks` "still gets an all-false summary and no tab", and that
closing it means widening the procedure's gate. As written that invites a future
widening of a `blocks.*` flag gate to close a gap that is effectively EMPTY:
that cohort is refused by `/apps/run` itself (which requires BOTH flags) and
cannot install, so all-false is the CORRECT tab set for them, not a deprivation.
Corrected in place.
F1 (perf) — NOT fixed, because it was MEASURED rather than reasoned about, and
the measurement inverts the analysis. The audit flagged `block_buzz_attribution`
having no index leading on `user_id`, correctly: seven live indexes, none on it.
But on prod that table holds ZERO rows / 72 kB, and the probe plans as a Seq
Scan costing 0.00 at 0.021 ms. Meanwhile the probe the audit CLEARED — scope
invocations, which does have `bsi_user_invoked_idx` — is the slower of the two:
863,305 rows / 314 MB, a Bitmap Heap Scan at 6.0 ms with 48 buffers read.
Neither is a merge blocker on a query cached for 60 s per nav. The index is
worth adding when that table grows; it buys nothing today, and adding an index
here is a manual, human-applied migration (CLAUDE.md rule 8), not a code change.
F3, F4 — left, deliberately. F3 (the store-gate ledger is narrower than its new
header) is bounded by whether the browser tier gates a merge, which it does not.
F4 (`live` gives each feed row its own 15 s interval) is real but harmless at
these volumes and pre-existing in shape.
Verified: unit 7,246 passed across 309 files (routers + blocks services + Apps);
component 78 passed on the three touched specs; prettier clean; typecheck 0
errors under src/. The 3 eslint errors on these files are PRE-EXISTING —
measured at branch HEAD via --stdin-filename, same count, same rules.
* test(apps): make the single-sourcing STRUCTURAL — my own guard was walkable
Audit round 3. Every finding was in MY round-2 fix, not in the product, and the
headline one was WALKED rather than argued.
F-A (guard) — the guard I added asserted (a) the string GLOBAL_SCOPE_ACTIVITY_OR
appears somewhere in blocks.router.ts, satisfied by the COMMENT the same commit
added, and (b) a 29-char literal is absent. Neither establishes that the probe's
`where` derives from the constant. The audit walked it in one edit: swap the
spread for a differently-spelled divergent predicate, leave the symbol alive in
that comment, update the router test's own literal — the edit a developer making
that change makes — and 67/67 stayed green while probe and feed diverged,
silently restoring the no-tab defect this PR exists to fix.
A guard on WORDS is satisfied by re-wording. Replaced with IDENTITY, in the
suites that can observe the actual Prisma call:
blocks.router.getNavSummary.test.ts expect(where.OR).toBe(GLOBAL_SCOPE_ACTIVITY_OR.OR)
user-app-surface.orchestration.test.ts same toBe on the feed's where.OR
`toBe` can only pass if the object handed to Prisma IS the exported one. The
file-scanning half is deleted rather than tightened — a stricter regex is the
same class of guard.
Proof, the audit's exact walk re-run: 1 failed where it was 67/67 green,
on this assertion's own message ("the probe did not pass the SHARED predicate
to Prisma — it re-spelled its own copy").
F-D (payload) — my `{ OR: Array<Record<string, { not: null }>> }` annotation,
chosen only to dodge a readonly-tuple error, erased a real compile-time check:
`Record<string, …>` does not constrain key names. Measured by the audit, and
re-measured here: `appBlockId` -> `appBlokId` inside the constant now gives
`TS2561 … 'appBlokId' does not exist in type 'BlockScopeInvocationWhereInput'`
at scope-activity-predicate.ts(41,10), where it previously typechecked clean.
Typing the array as Prisma's own input restores it and compiles — the loose form
bought nothing.
F-B + F-C (payload/scaffolding) — putting the constant in
user-app-surface.service.ts dragged that heavy service into blocks.router.ts's
STATIC import graph, which is exactly what that router's five `await import(…)`
sites exist to avoid, and which four router suites' one-key vi.mock factories
would have tripped over on the next nav-summary case. Both close with one move:
the constant now lives in its own leaf module whose ONLY import is type-only.
Pinned by a test that classifies every import in that file.
Verified: unit 7,246 passed / 309 files; the three predicate suites 67/67;
prettier clean; typecheck 0 errors under src/ AND red on the planted typo. The
2 eslint errors on blocks.router.ts are PRE-EXISTING — same count at HEAD via
--stdin-filename; my two new files are 0/0.
|
||
|
Zachary Lowden
|
33f4dc3f69 |
feat(apps): consolidate the build funnel into one state-aware /apps/build (#4685)
* feat(apps): consolidate the build funnel into one state-aware /apps/build Three sub-nav items and two pages become one. `Build apps` (/apps/get-started), `Create` (/apps/submit) and `My apps` (/apps/mine) were largely one another's content: get-started was 100% static marketing whose every CTA pointed off-platform, and /apps/submit's on-platform branch was the same copy-paste wall a second time. Only /apps/mine did real work. A developer's path through them was three tabs that mostly showed each other. /apps/build replaces them with one route in three states: A pitch - not an author: the recruiting page + an access CTA B first-app - author, nothing yet: quickstart + create C workbench - author with apps or submissions: the app list + New app /apps/get-started and /apps/mine 301 to it (/apps/my-submissions repointed straight there too, so there is no 301 chain). /apps/submit KEEPS its route - getOwnerEditHref deep-links every offsite listing at /apps/submit?edit=<id> from the store card and the listing-detail menus - it simply loses its tab. ONE PREDICATE, TWO CALLERS. canAccessAppsBuild (shared/utils/app-blocks-access) is called by both the SUB_NAV_LINKS row and the page's getServerSideProps. The defect this exists to prevent has shipped twice - a tab offered to a cohort whose page answers notFound (#3899, and again as a deploy-blocking finding on #4668) - and both times the two rules were written separately and drifted. Also closes a third instance of that defect that was documented and deliberately left open: Marketplace was `visible: () => true` while /apps gates on resolveAppsPageAccess. It could not be fixed before because closing it dropped the get-started-only cohort to one tab and the <2 collapse deleted their bar. Build being store-gated removes that objection. Adds the first instrumentation this funnel has ever had (one AppsBuild_Action type carrying its step in details, plus the ClickHouse Enum16 widening it needs, to be applied by hand before deploy). Verified: unit tier 38558 passed; the four AppsSubNav browser suites 93/93 under a PLAYWRIGHT_BROWSERS_PATH shim; typecheck clean on both configs for every touched file. The new sub-nav/page agreement guard was measured RED at origin/main with a byte-identical file and green here. * fix(apps): carry the POST-APPLY tracker-restart marker on the new actions migration Caught by a MERGED-TREE run, not by either side on its own. `main` added `action-type-enum-drift`s marker requirement after this branch was cut; the two changes touch no file in common, so both sides were green and the merge was red. Applying the DDL without restarting civitai-clickhouse-tracker ships a type that collects zero rows while every signal says it worked. * fix(apps): round-2 audit fixes — widen the gate ledger, compare the Build row to its page, correct three voided claims F1 (downgraded to prose-only). The audit called the OWNER_SUBMISSIONS_URL repoint deploy-blocking. It is not: measured in Flipt v2 env `civitai-app`, `app-blocks-author`, `app-listings` and `app-blocks-enabled` all roll out to the SAME two segments (`moderators`, `app-dev-testers`), so `appBlocksAuthor` implies `hasAppsStoreAccess` and the `{isAuthor, no store access}` cohort is empty by construction. Routing unchanged. What WAS wrong is the prose: - comment.notifications.ts: the decision record still argued `/apps/mine` was better because it needed "the developer cohort rather than the narrower store-access one". `/apps/build` requires store access as a hard AND, so that argument is void — and it inverts: on the cohort axis the destination is now NARROWER than the public detail page. Rewritten to say the destination is chosen for CONTENT (state C is the owner's submissions table) rather than reach, and the superseded measurement is kept but labelled as no longer discriminating. - app-listing.notifications.ts: replaced "every App-Blocks flag is staged mod-only and a moderator holds all of them" with the actual segment proof, plus what would break it (widening one flag's segments without the other) and the live counter-example that shows they can diverge (`app-listings-public-external` already uses `testers`). F2. appsBuildGateCallSites SCAN_ROOTS was ['components/Apps', 'pages/apps'] — the sibling ledger's PRE-FIX value — so its "fails when the set GROWS" claim was blind outside two directories. Measured: a third module calling canAccessAppsBuild at components/AppLayout/AppHeader/auditThirdGate.ts SURVIVED the whole node tier (9/9 green). Widened to ['components', 'pages']; the floor moved 40 -> 1200 so narrowing the roots back cannot pass silently, and a new guard names the two blind-spot directories explicitly. F3. subNavRowsMatchPageGates claimed to evaluate EVERY row against its page's gate and evaluated one. The Build row was compared to nothing: both `(_s, c) => c.isAuthor` and `(_s, c) => c.canSeeStore` SURVIVED the blocking tier, the latter being #4668 reintroduced verbatim. Added the Build loop against the real resolveBuildPageAccess across (store x author x getStarted), plus a per-term positive control. Docstring narrowed to the two flag-gated rows it actually covers, with why the other four have no second copy to drift from. F5. feature-flags.service.ts still promised widening `app-blocks-get-started` to ['public'] was "a one-line flag change". Post consolidation the pitch sits behind `canAccessAppsBuild`, so widening it alone gives the new cohort a notFound from /apps/build, no user-menu entry and no sub-nav — the pitch unreachable by every route. Names the new precondition. F7. cliCommands.ts cited apps-my-submissions-redirect.test.ts, which this PR deletes. Repointed to apps-build-redirects.test.ts. F8. CopyableCommand's onCopy doc said "Fired on a successful copy"; handleCopy calls copy() then onCopy?.() unconditionally and Mantine surfaces no success signal. Corrected the doc rather than the code, and said why. Mutation matrix (node `unit` tier, pre-fix tree = |
||
|
briant
|
8a39b7c876 |
build: consume form-graph from npm instead of the local link
Swaps the pnpm link: override for the published package (^0.3.0, currently 0.3.1) and removes the two accommodations the link needed: the turbopack.root widening (Turbopack refused files outside the repo) and the tsconfig react/react-dom paths pin (the linked checkout's own @types/react could split the program). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
briant
|
60004ec26f |
feat(generation): 14 more image families, reconcile lane, persistence scopes
Graphs + handlers + differentials for flux (5 modes), flux-kontext, flux2, flux2-klein, boogu, krea2, imagen4, pony-v7, reve, mai, ernie, seedream, anima, mage-flow, hi-dream, hi-dream-o1, openai, lens, qwen family and nano-banana — the image hub is complete except grok (spans both output types). Probed workflowVersions semantics differ per family: boogu's model wins the workflow (reconcile registry), mage-flow index-remaps (a correct), qwen substitutes via the lock. reconcile.ts: v1's cross-level checkpoint effects as one pure policy with two adapters — a raw normalizer before parse and a store rule on both hubs. Locked slots beat cross-family models; version siblings re-pick through the lock. Family effectiveEcosystem computeds delegate to it. Persistence: per-family scope buckets via the lib's new graph.scope (familyScope group-or-key, workflowScoped images/video, perModelScope turbo-variant cfg/steps, bare seed/controlNets, ecosystem per output type), persisted under one localStorage record in BaseGenerationForm; layout and rehydration pinned by persistence.test.ts. Review-pass hardening: handler differential pins each case's parsed ecosystem (caught four redirect-vacuous cases), matrix extra-shapes keyed by record with canaries, per-key boundary shape, exact-set video canary (exposed legacy WanVideo as row-less). Reuse hoists: familyResources, perModelSlider, img2imgImages, versionModeOf shared by graph+handler lanes, resourcesToLoras. Types: zero as-never casts. Fixed a mis-transcribed snippets reference schema (now imports the real one), widened PartialResourceValue to the picker's real pre-hydration contract, def factories converted to satisfies so input types flow to Controller onChange, SelectInput/OutputFormatInput genericized over their value unions. Toolchain (remove before PR): link: override + turbopack.root widening + react type-path pins for local lib development. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
briant
|
a961f68bfc |
feat(generation): form-graph client forms, handler lane, and seedance
Three lanes on top of the ported graphs, all review-driven this session:
CLIENT (`src/components/form-graph/generation/`):
- BaseGenerationForm owns the store over the composed root (FormProvider),
renders the workflow picker and the submit footer, and switches between
the per-output forms on the graph's own `output` computed.
- Image/VideoGenerationForm in the generation_v2 GenerationForm idiom: one
`<Controller graph={imageHub|videoHub} name=...>` per field — typed from
that hub's registry (computeds and branch tags included via the state
type) — wired to the REAL generation_v2 inputs: BaseModelInput,
ResourceSelect* with a ported VersionGroupSelector,
ImageUploadMultipleInput, VideoInput, GenerationTextEditor (snippets +
trigger words from editor meta), ControlNetsInput, AspectRatioInput,
Priority/OutputFormat, ActiveWildcards with the ported add/remove flow,
ResourceAlerts via MultiController, and v2's wan version picker (sets
ecosystem; the tag follows).
- Demo at `/form-graph` (standalone; mounts ResourceDataProvider itself).
`/data-graph-v2` now renders the packaged GenerationFormV2 — its
hand-assembled provider stack was missing GenerationProvider and
ResourceDataProvider and threw for any signed-in visitor.
SERVER (`src/server/services/orchestrator/form-graph/`):
- Per-family handlers (ltx, wan, seedance, stable-diffusion, z-image,
chroma) transcribed from the data-graph handlers but importing only
ported modules, plus a dispatcher (`createFormGraphStepInput`) with v1's
seed normalization and enhanced-compatibility engine rewrite; unported
ecosystems are a loud error.
- The acceptance gate: a differential test feeds BOTH dispatchers the same
parsed data and asserts identical @civitai/client steps — 29 cases over
draft batching, comfy routing, controlnets, every wan version (both
flipt states of 2.2 multi-step), distilled LTX, prompt-enhancer $refs.
GRAPHS:
- video/seedance.graph.ts (no resources, no negative prompt; per-version
resolution/duration ceilings) — unblocked the video matrix's
hidden-ecosystem gate coverage.
- `resolveCompatibleEcosystem`: v1's workflow→ecosystem sync effect as a
pure redirect in both hubs' input transforms (txt2img + WanVideo30
parses as SD1, pinned), with the workflow-group override guard.
- LTX gets the model-wins split (`effectiveEcosystem` emit +
`checkpointDef({ modelWins: true })`); wan/ltx `shift`/`numFrames`
mirror v1's REJECTING inputs; full v1 picker meta everywhere
(controlNet groups, sampler presets, gate payloads, editor meta with
the editor-vs-registers-target split for ZImage Base).
- Family dispatches are switch statements returning member graphs
directly (record-less branches).
Toolchain: the form-graph override is `file:` (Turbopack cannot read
outside the workspace root, which `link:` always violates) and the package
is in `transpilePackages`. Requires local form-graph at 066377d — publish
+ swap the override before any push or PR.
Verified: 4,757 differential cases, typecheck and lint clean; full unit
suite 29,056 passed with only the 10 pre-existing Windows-portability
failures (identical on clean main).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Zachary Lowden
|
eaa277b0d4 |
docs(build): close option 1 on the Turbopack chunk collision — nested async chunking does not fit the memory ceiling (#4500)
The flag was already tried and reverted: #3458 enabled it 2026-07-30, #3807 turned it off 08-11 after the release build OOMKilled three times at 37-39 GiB against the enforced 40Gi builder cgroup. The 2026-08-18 doc was written after that revert and still recommended it as the only lever, with the RSS cost listed as unverified. Re-measured on 16.3.1: +43% peak next-build RSS, worse than the +33% on record. Live distribution n=67 builds, worst observed 28.59 GiB; times that multiplier lands at 37.3-39.8 GiB, the band the OOM kills were recorded in. No headroom. The turbopackSourceMaps escape hatch is closed too - it cannot be split client/server and server .js.map has a hard consumer gate. The flag VALUE is unchanged (still false). This is docs plus one comment, so the lever is closed off in the place someone would next reach for it. |
||
|
Zachary Lowden
|
bc74ba06ea |
feat(apps): merge /apps/my-submissions into /apps/mine — one author table, nested history, inactive collapse (#4154)
* feat(apps): merge /apps/my-submissions into /apps/mine — one author table `/apps/mine` becomes THE author surface: one table over every app the caller owns or holds an accepted collaborator seat on, on-site and off-site, with that app's publish history nested in its row and fetched on expand. `/apps/my-submissions` 301s here; its page component is deleted rather than stubbed. The row set stays `appListings.listMine` (→ `resolveAccessibleListingIds`). Re-deriving it from a submissions read is the regression this page exists to prevent: a publish request is scoped to `submittedByUserId`, so a collaborator (who submitted nothing), a transfer recipient and a moderator-claimed owner all see an empty page on apps they can plainly edit. Server: - `listMyAppListings` now carries `iconUrl` / `coverUrl` / `updatedAt`. The media columns are integer FKs to `Image`, so this is a select widening plus a shared URL projection (`listing-media-url.ts`, also adopted by the public store read so there is one copy of the widths). No screenshot fallback here — a missing cover is the fact its author needs to see. - New `appListings.listingHistory`: one listing's merged publish history, authorized through `resolveListingAccess` (ownership ∪ accepted seat), gated on `app-blocks-author` only. UI: - Active/inactive partition on LISTING status only (`rejected`, `removed`). `draft` stays in the main table; a withdrawn SUBMISSION never moves an app. - Inactive collapse: closed by default, count in the header, paginated, with `aria-expanded`/`aria-controls` derived from the same state the panel renders. - Icon + cover per row at fixed dimensions with `loading="lazy"`, and a placeholder for each — all 11 `removed` listings in production have no cover, so that path is the main render path for the collapse. - Mobile degrades to cards; exactly one layout is rendered. - The "My submissions" sub-nav tab is retired and `hasSubmissions` now widens "My apps", so no population loses its route. * test(apps): use the canonical dbMock in the two new block-service tests `no-direct-shared-module-mock` went red on them: a per-file mock of the db-client specifier freezes that file's mock shape into every later file in the same worker under `isolate: false`. Both now take `dbMock.dbRead` / `dbMock.dbWrite` from `src/__tests__/mocks/db.mock`, which the global setup registers and resets per file. No assertion changed — the local `mockDb` / `mockWriteDb` handles keep their names, so the bodies are untouched. The guard is a TEXT scan over the file, so the forbidden call cannot be quoted even in a comment; the note explaining the migration says so rather than spelling it. * fix(apps): surface first-version submissions, restore the completeness advisory Four audit findings on #4154. §1 (blocking) — `app_block_publish_requests.app_block_id` is NULL until approve, and the history query keyed on it alone. Measured on production 2026-08-20: 3 of 3 `rejected` and 27 of 33 `withdrawn` rows carry a NULL FK, so 100% of rejections were unreachable — including from the "your app was rejected" notification this PR repoints at /apps/mine. Two sub-cases: (a) a PENDING first version still has its draft listing, only the FK is null. `blockRequestWhereForListing` now falls back to the listing's slug, scoped to its CANONICAL OWNER (never the viewer, so a collaborator still sees it; never unscoped, so a recycled slug cannot leak a stranger's review) and gated on `kind === 'onsite'`, which is now what keeps off-site listings out of the code stream. (b) a REJECTED/WITHDRAWN first version has its listing deleted to release the slug. New submitter-scoped `listMyOrphanedSubmissions` + `appListings.listMyOrphanedSubmissions` surface those as their own always-visible "Submissions without a listing" group, with the reviewer's reason. Submitter-scoped because it is the only identity the surviving record carries. No schema change, no migration; the deletion is deliberately left as-is. §2 (blocking) — `computeListingProblems` had lost its only renderer when the two my-submissions tables lost their importer, which also falsified `listingCoverUrl`'s "the author must see the gap" rationale. `listMine` now carries `problems` and the row renders `ListingProblemsIndicator`. §3 — Withdraw was offered to everyone. Both procs are submitter-scoped, so a collaborator / transfer recipient / mod-claimed owner got a button that only red-toasts. Entries now carry a server-computed `canWithdraw`. §4 — `blocks.withdrawPublishRequest` carries `enforceAppBlocksFlag` while this page gates on `appBlocksAuthor` only. The UI is gated rather than the mutation loosened: removing an authorization gate is not a change to make in passing. Also fixed here: the empty state was an early return, so an account whose only records were orphans would have seen "you don't own any apps yet" instead of them — the same defect one layer up. Caught by its own test. `app-access.accessible-listings.test.ts`'s fake discriminated its two `findUnique` reads by `select.slug`, which worked only because the access resolver happened not to want that column; it now keys on `connectClientId`, which is asked for by exactly one of them on purpose. * fix(apps): close the seams an independent mutation sweep found open My sweep reported 9/9 killed; an independently-constructed one found 7 survivors out of 13. Two of them reintroduced, undetected, findings this PR had just fixed. That is the standing lesson about a sweep built from its own assertions, and it is why these are behavioural fixes, not extra assertions. (a) The off-site half of `canWithdraw` had no guard. The two streams are mapped by two separate `.map()` calls, so the wiring exists twice and can be wrong in one alone — rebinding the LISTING stream to treat every viewer as the submitter SURVIVED, while the identical block-stream mutation died. Each stream is now pinned on its own, plus a case asserting both answers in ONE payload with the two ids deliberately different. (b) The `problems` wiring was unpinned server-side: `computeListingProblems` is well tested as a pure function and the client test supplies `problems` by hand, so both sides were green in isolation and nothing tested the join. `problems: []`, `coverId` fed from `r.iconId`, and the screenshot default `0 -> 99` all passed. Fixtures now carry pairwise-distinct sibling values (iconId 7, coverId 9, screenshots 3) so an operand swap changes the answer. The screenshot mutant needed one more turn: `0 ?? 99` is `0`, so with the relation PRESENT the default is unreachable by construction. It fires only when `_count` is absent, which now has its own case — and pins the direction, since a default that hides `no-screenshots` silently turns an incomplete listing into a clean one. (d) Two silent-zero paths on the one surface this population has. A failing `listMine` early-returned and took the orphan group with it; and `orphansQuery.error` was read NOWHERE, so a failed orphan read rendered nothing and said nothing — indistinguishable from "you have no rejected submissions", the exact lie the group exists to stop telling. Both now surface, neither blanks the other, and the empty state no longer renders over a broken read. Also, all cheap and all wrong before: - the kind-gate rationale claimed a no-op; it is a real behaviour change for the offsite-listing-that-carries-a-block shape (#3844), where the old `appBlockId ? … : null` DID run the block query. Stated as the tightening it is, with why. - a `describe` title my own delta falsified ("CONDITIONAL on the block, not on the kind" — it is now conditional on the kind). - `take: limit` ran BEFORE the ownership de-dup, so the orphan group could return a short page while more existed. Split into `ORPHAN_SCAN_LIMIT` (the read) and `ORPHANED_SUBMISSIONS_LIMIT` (applied after the filter). The dead `opts.limit` — unreachable from the router — is dropped rather than wired. - softened the null-FK branch's coverage claim: it does NOT restore first-version history for a transferred or moderator-claimed listing, only for an owner who submitted it themselves. * fix(apps): close the two guards that became silent-zero paths themselves F1 — the early return keyed on orphan DATA length but not orphan ERROR, so with both reads failed and zero orphan rows only the listMine alert rendered and the orphan failure reported nothing, permanently. Zero rows is not the same fact as "the read succeeded and found none" — the silent-zero lesson, applied to the guard written for it. F2 — the container never forwarded `orphansQuery.isLoading`, so `hasNothingAtAll` could not tell "no orphans" from "orphan read in flight". The two procedures batch into one request under `httpBatchStreamLink` but stream back independently, so `rowsQuery` resolving empty first is ordinary, not exotic — and the page asserted "You don't own or collaborate on any apps yet" over a pending read. Streaming makes that MORE reachable, not less. The `isLoading && orphanedSubmissions.length === 0` guard was also uncovered (reverting it to plain `isLoading` survived the last battery). Its real effect is not the error case its comment described: once orphans resolve while rows are still loading, falling through renders the group rather than a spinner over data that already arrived. Covered and documented as what it does. Two comments corrected, both over-claiming rather than wrong: - the de-dup ordering test claimed it kills a `take` mutant. It does not — `mockImplementation` ignores `take`, so all 28 rows come back regardless and the case passes. It pins the ORDER of de-dup vs display cap; renamed and re-commented to say so, and the remaining self-referential `take` coverage is recorded in the follow-up issue. - `ORPHAN_SCAN_LIMIT` read as though under-return were closed. It is 4× harder, not closed: still reachable at ≥76 of the newest 100 rows de-duping as owned with non-owned rows past position 100. Stated with the real bound, plus the index reality — neither existing index can serve the ORDER BY under this predicate, so Postgres top-N sorts either way and a wider scan is cheap. |
||
|
briant
|
0f1830eb56 |
Merge remote-tracking branch 'origin/main' into moderator-app-pages
# Conflicts: # src/pages/api/mod/csam/training-data-report.ts # src/pages/api/mod/training-data/resolve.ts # src/server/controllers/csam.controller.ts |
||
|
Zachary Lowden
|
c78f85dd5a |
fix(deps): Next 16.3.0 → 16.3.1, and make the compiled-branch gate hard (#3983) (#4075)
* fix(deps): Next 16.3.0 -> 16.3.1, and make the compiled-branch gate hard (#3983) This is the fix for #3983. The defect was in the bundler, not in our source. Turbopack's value analyzer in 16.3.0 models a bare `return someAsyncFn()` tail call as `Promise<Promise<T>>`. That is always truthy, so a caller that `await`s it is analysed as always-true and every statement after the resulting conditional is eliminated as dead code. `isAppListingsEnabled` ends in exactly that shape, which is why `resolveStoreVisibilityScopeUninstrumented` lost two of its three returns, fell off the end, and produced `undefined` for every non-privileged caller — served as the whole catalog on one read path (`?? 'full'`) and as an empty store on the other (`?? 'none'`). Upstream: vercel/next.js#96601 "[turbopack] Collapse nested promises in the analyzer", backported as #96675, shipped in 16.3.1. MEASURED, not inferred. Two production builds of THIS commit on one machine, same Node 24.19.0, differing only in the pinned Next: 16.3.0 async function S(e){if(await p(e))return"full"} 16.3.1 async function w(e){return await c(e)?"full":await y(e)?"public-external":"none"} Both read out of the emitted `.next/server` chunks by source-map attribution and identified by their source neighbour `STORE_SCOPE_FLAGS`, never by minified name. Note the fixed form is a TERNARY — `grep 'return"public-external"'` returns zero on the FIXED build too, which is why the gate reads source maps. `package.json` already allowed 16.3.1 (`^16.3.0`); only the lockfile pinned 16.3.0, so the substance here is the lockfile. The floor is raised to `^16.3.1` so a fresh resolution cannot land back on the broken compiler. `patches/next@…` is renamed and its `patchedDependencies` key updated — that patch is the unrelated libvips/SVG one-liner (vercel/next.js#96681), it still applies cleanly, and 16.3.1 still does not carry the loader entry upstream, so it stays. `--warn-only` is removed from `scripts/assert-compiled-branches.mjs` in the same commit. It existed only because the 16.3.0 build genuinely violated the gate, and a permanently-red gate trains everyone to click through. Keeping the bump and the strictness atomic means the gate's strictness always matches the toolchain: a revert of the bump turns it red instead of silently passing. Verified on this commit: - gate exit 0 (hard, no --warn-only) against the 16.3.1 build - gate exit 1 against a 16.3.0 build of the same tree — watched red - `scripts/ci/assert-next-svg-patch-applied.mjs` OK on both installed copies - unit suite 1149 files / 18,123 tests passed, 0 failed Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(build): ship @swc/helpers' module-sync branch into the standalone image (#3983) The bump built clean, passed every source-level gate — unit suites, typecheck, ESLint + Prettier, schema drift, the event-engine pin, and the now-hard compiled-branch gate — and the container could not boot: Error: Cannot find module '.../@swc/helpers/esm/_interop_require_default.js' at ... next/dist/server/require-hook.js code: 'MODULE_NOT_FOUND' Same shape as the defect this PR exists to fix: a correct source tree producing a broken artefact, invisible to everything that reads source. ROOT CAUSE, measured on the published artefact rather than inferred. `output: 'standalone'` does not ship node_modules; it ships the subset @vercel/nft traced. nft resolves a bare specifier under the `require`/`default` conditions. Node (>= 22.10) additionally honours `module-sync` for a CJS `require`. When a package's `exports` map points those two at different files, the build traces one and the running process asks for the other. next/dist/shared/lib/constants.js does `require('@swc/helpers/_/_interop_require_default')`, reached from the generated server.js via `next` -> config.js -> constants.js, i.e. before any application code. The relevant delta is not next itself but next's own dependency: next 16.3.0 next 16.3.1 @swc/helpers 0.5.15 0.5.23 ./_/_interop_require_default {import,default} {module-sync,webpack,import,default} require.resolve() under CJS cjs/...cjs esm/...js Both resolutions were RUN, not reasoned about. nft still traced the cjs file, so the published image carried that package as exactly cjs/_interop_require_default.cjs, cjs/_interop_require_wildcard.cjs and package.json — no esm/ directory at all. Adding only the missing esm/ directory to that exact image, nothing else changed, boots it: "Next.js 16.3.1 ... Ready". FIX. `outputFileTracingIncludes` force-includes BOTH condition branches of EVERY installed @swc/helpers copy — not the one file missing today, because which helper Next requires and which branch each resolver picks are upstream details that move. Globs are version- and hash-agnostic (`@swc+helpers@*`), plus a flat form for a hoisted layout. ~950 KB per copy. Verified on a local production build of this commit: both copies land in .next/standalone with complete esm/ (108 and 105 files) and cjs/, and next's virtual store links the 0.5.23 copy. Attached to three existing API-route keys rather than a `'**'` key. copyTracedFiles unions every entry's traced set into the single .next/standalone node_modules, so one entry carrying it is enough, while `'**'` would make all 572 entries read/parse/rewrite their .nft.json concurrently — 826 MB of JSON in one Promise.all — on a build already tuned against OOM. GATE, because a glob is a silent no-op once it stops matching. scripts/ci/assert-standalone-boot-graph.mjs runs in the Dockerfile's RUNNER stage: the first gate in that file to run against the runtime filesystem rather than the build tree, and the only one that can see this class of defect. It reads the GENERATED server.js for the specifiers that process requires at module scope and loads them in a child rooted at the shipped tree — no package, version, virtual-store path or patch hash hardcoded, so it keeps covering this after the next bump. Exit 2, never 0, when it cannot observe its input. It must run in the runner and not the builder: /app there is byte-for-byte what ships, whereas the builder's complete node_modules sits above .next/standalone on the resolution path and can satisfy a require the image cannot. Watched red and green on real artefacts, not only fixtures: - exit 1 with this exact MODULE_NOT_FOUND against the published broken image; - exit 0 against the same image with only the esm/ directory added; - exit 0 against the local production build of this commit, isolated from any parent node_modules; - exit 1 again after deleting exactly esm/_interop_require_default.js from that same local build. src/tests/build/standalone-boot-graph.test.ts pins the MECHANISM (a module-sync/default split with only the default branch present) rather than the package, and all 5 of its cases were watched to fail against a neutered gate. NOT VERIFIED. Nothing about production: this is only true of production once it merges, is promoted main -> release, is built and is serving. The gate covers the ENTRYPOINT's require graph; route chunks load lazily, so a condition mismatch reachable only from a route would still surface at request time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: retrigger preview The previous preview run (pr-preview-4075-b2wnw) never scheduled: its build-image and typecheck pods sat Pending with ExceededNodeResources for 82 minutes and the run hit the 1h30m PipelineRunTimeout. No verdict was produced — this was build-pool capacity contention, not a code failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Zachary Lowden
|
fa001acecb |
docs(build): diagnose the intermittent Turbopack chunk-hash collision (#4084)
Documents the root cause of the intermittent 'Two or more assets with different content were emitted to the same output path' build failure, and leaves a pointer comment at the one config flag that would mitigate it.
Root cause: a birthday collision in Turbopack's 7-character server chunk-name hash. The build emits ~24.5k server chunks; the first hash character is bounded in practice to {0,1,2}, leaving roughly 5% of nominal hash space, with ~11.7k chunks in the largest namespace.
Ruled out by experiment: the shared BuildKit cache mount. The exact failing tree reproduces the failure locally with no Docker, no cache mount and no concurrency, twice, identically.
Note on the error text: the two hashed files it prints are Turbopack's debug dumps, not the collision; the colliding path is the line above. '2 errors' is one collision reported twice (chunk plus source map).
No code fix here. The only lever that attacks the mechanism (turbopackServerSideNestedAsyncChunking, ~-71% chunks, ~9x fewer collisions) is broken on 16.3.0 and is deliberately disabled for a peak-memory cost that is unverified.
Anti-trap note recorded in the doc: a Next 16.3.1 bump makes the failure go away by re-rolling which chunk pairs collide, not by changing collision probability. Since another PR already lands 16.3.1 for unrelated reasons, that bump will look like the fix and the failures will return.
|
||
|
manuelurenah
|
23e706a225 | Merge branch 'main' into moderator-app-pages | ||
|
Justin Maier
|
03979077a0 |
perf: sever client→server import bridges in the _app graph (#3802)
Removes 95 modules from _app's client module graph: no Prisma client, no redis client, no server env schema compiled into the browser. Adds a convention guard (test:lint-rules) that fails if _app's graph reaches server infrastructure again, printing the offending import chain. Also fixes a pre-existing durable-logout path: _app no longer deletes the auth cookie on an ambiguous session: null, since getServerAuthSession fail-softs and a hub outage is indistinguishable on the wire from an expired token. Adds the regression test that path never had. Measured negative, stated in the PR: module-graph size is NOT a meaningful lever on peak builder RSS (-2.8% for ~7% of the graph). This does not buy back the server-chunk regression from #3807. Reviewed adversarially in four rounds; ~15 findings addressed. |
||
|
Justin Maier
|
c771513011 |
fix(build): bring release-build peak memory back under the 40Gi builder limit (#3807)
* perf(build): stop nested-async chunking the server build
The flag trades peak builder memory for smaller server chunks: ~72% fewer
emitted server chunks and roughly half the server chunk bytes, in exchange for
~+8% CI build time and ~+33% peak builder RSS (numbers from #3458, recorded in
the comment this replaces).
That trade was accepted when nothing measured build RSS. It now has to be paid
against an enforced 40Gi builder cgroup, and the release build OOMKilled three
times at 37-39 GiB. Turning it off is a real regression in emitted chunk size,
not a free win, but a build that cannot complete is worth less than smaller
chunks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(build): pin turbopackFileSystemCacheForBuild to the pre-16.3.0 default
Next 16.3.0 flipped this experimental flag's default to true, so leaving it
unset silently changed the production build. `turbopack-build/impl.js` derives
`dependencyTracking` from it, which changes what turbo-tasks retains in memory,
not only what is written to `.next/cache`.
The 16.3.0 upgrade (
|
||
|
briant
|
1d513423b5 |
Merge origin/main into moderator-app-pages
54 commits. Three conflicts: - `package.json` — both sides added a workspace dependency (`@civitai/moderation` here, `@civitai/flipt` on main). Kept both. - `pnpm-lock.yaml` — took main's and regenerated with `pnpm install --lockfile-only` against the merged package.json rather than hand-merging it. - `src/pages/moderator/reports.tsx` — modify/delete. Our delete stands: the page is migrated and `reports` is already in the MIGRATED_ROUTES redirect in `src/pages/moderator/[...slug].tsx`. Main's schema changes left the generated Prisma client stale (`userCosmeticShopItemResale`, `sfwBio`), so `pnpm run db:generate` was re-run; the regenerated output is included. Root typecheck 0 errors, apps/moderator typecheck 0 errors. ⚠️ The deleted page was not just deleted on main — `159f535f7a` ADDED a capability to it: `RemoveReportedPlacement`, the removal path for the new `StickerPlacement` report reason. The migrated `/reports/[slug]` in apps/moderator knows the reason (it is in `reportReasonLabels`) but has no placement removal action, so that capability is currently absent from the spoke. Recorded rather than ported here — it is a feature, not a merge resolution. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
briant
|
d9ffacb259 | Merge branch 'main' of https://github.com/civitai/civitai | ||
|
briant
|
0f5f28b90d |
feat(packages): add @civitai/flipt and wire creator-studio to it
Extracts the monolith's Flipt client into a shared workspace package so other apps can gate on the same flags. The package carries over the production hardening as-is — init timeout, failure circuit breaker, generational TTL eval cache, dev-only local overrides, fail-closed on every path — and follows the @civitai/axiom / @civitai/redis shape: raw TS entry, package-owned lazy env, injected logging. Flag keys stay in the app. FLIPT_FEATURE_FLAGS and the cache-bypass set remain in src/server/flipt/client.ts and are passed to the factory as `cacheBypass`; which flags are incident kill-switches is app knowledge, not package knowledge. Connection env resolves on the first evaluation rather than at construction, so a missing FLIPT_URL degrades that instance to fail-closed instead of throwing out of the import that built it. Building it eagerly broke 61 unit suites at collection, whose env mock has no Flipt vars. creator-studio gets a getFlipt() shim mirroring its redis.ts/logger.ts (lazy, globalThis-cached, app logger injected), the transpile + dep entries, documented env, and a test pinning the wiring — nothing else in the app imports the shim yet, so without it neither svelte-check nor a build would prove it resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
manuelurenah
|
f108d51493 |
fix(prisma): externalize @prisma/client so $queryRaw sees one Sql class
`Prisma` imported from `@prisma/client` was bundled into the app layer while
`dbRead`/`dbWrite` reached a second copy through the transpiled
`@civitai/db-schema`. `$queryRaw` identifies its tagged-template argument with
`instanceof Sql`, so a `Prisma.join()` built by the other copy failed that check
and was bound as a plain value, producing:
42883 operator does not exist: integer = jsonb
This 500'd every `homeBlock.getHomeBlock` on the homepage feed, and was latent
for the ~20 other server modules that use `Prisma.join`. `require.resolve`
reports a single physical path from every workspace dir, so the duplication is
only visible at the bundler layer.
`@prisma/client` is already in Next's built-in serverExternalPackages defaults;
listing it explicitly is what stops `transpilePackages` from pulling it back in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
briant
|
e27f4a72ee |
Merge remote-tracking branch 'origin/main' into moderator-app-pages
# Conflicts: # CLAUDE.md # apps/moderator/CLAUDE.md |
||
|
Zachary Lowden
|
30e32db0a9 |
feat(telemetry): emit structured logs over OTLP alongside Axiom (dark by default) (#3736)
* feat(telemetry): emit structured logs over OTLP alongside Axiom (dark by default)
Adds a third sink to the single `logToAxiom` chokepoint: the same record is
handed to the OpenTelemetry Logs API in addition to the existing stderr write
and the Axiom ingest. Neither existing sink changes behaviour.
Ships INERT: `emitOtelLog` no-ops unless `OTEL_LOGS_ENABLED === 'true'`, which
nothing sets. Merging this changes nothing observable; enabling it is a config
change per workload, not a code deploy.
The bridge (`@civitai/telemetry/otel-logs`)
- Injected into `createAxiomLogger(overrides, deps)` rather than imported by it,
so `@civitai/axiom` keeps no OpenTelemetry dependency and no OpenTelemetry
semantics. Its own env.ts header already prescribed this shape.
- ONE serialization, N sinks: the line is built once and the OTLP body is that
exact string, so a query written against either path matches the other and the
hot path pays for one JSON.stringify. This refactors the three-branch stringify
catch ladder from three console.error calls to assign-once/write-once; a test
pins each branch.
- Severity: `type` is a hybrid field (mostly a severity word, sometimes an event
name, sometimes absent), so only the four clear cases map. Everything else is
UNSPECIFIED(0) with severityText OMITTED — downstream level resolution prefers
text over number, so an invented string would outrank a correct number. The raw
value always survives as the `civitai.type` attribute, so nothing is lost. A Map,
not an object literal: an object lookup with a caller-supplied key fails open on
inherited keys like `toString`.
- Attributes are a CLOSED set. Spreading the payload would reproduce the schema
explosion `safeError` exists to prevent. The payload is already in the body.
- The logger is resolved LAZILY per emit and only once a provider is registered.
`logs.getLogger()` called before registration returns a proxy that is never
delegated and stays silent forever, with no error — a module-scope getLogger is
the bug, not a style choice.
- Counters make a zero legible: a bare "no records arrived" cannot distinguish a
disabled flag from an unregistered provider from a broken emit. Emitted and
skipped{disabled|no_provider|emit_threw} separate them.
- Emission can never throw to the caller: logToAxiom is awaited on hot paths.
SDK correctness
- Batch processor: `exportTimeoutMillis` made explicit (env-tunable) because it is
the one value coupled to something outside the process — a hung exporter must
not hold the shutdown flush past the termination grace period. The default is
sized for the shortest grace period any of these workloads runs with.
- Shutdown was SIGTERM-only and awaited NEITHER promise, so the final batch raced
termination — immaterial at one record per boot, a real loss path once it carries
traffic. Now SIGTERM + SIGINT, awaited, idempotent across a second signal, and
allSettled so one failing exporter cannot block the other flush.
- Provider construction moved into the package so the `traceBased` hazard has one
home and a regression test. `traceBased: true` would drop every record emitted
inside an unsampled span; with head sampling below 1.0 that is most in-request
records, silently. It is inert at SDK defaults and must stay that way.
- `serverExternalPackages` gains the four logs-pipeline packages so the Logs API's
globalThis provider registry is written and read by one module instance.
`@opentelemetry/resources` and `semantic-conventions` are deliberately excluded —
the web tracing SDK reaches them from the client graph.
Tests
41 tests across the two package suites, using the real SDK — no module mock, which
would make the assertions vacuous. Each guard was watched to fail for its own
reason: 14 mutations, each killed by the specifically-named test that owns it.
The bundling failure is structurally invisible to Vitest (it aliases the package to
source and never runs the bundler). The runtime check for it is
skipped{reason="no_provider"} being non-zero in a real process; a build-time
module-identity scan is the follow-up.
* fix(telemetry): drop the serverExternalPackages additions from this PR
The four logs-pipeline entries (@opentelemetry/api, /api-logs, /sdk-logs,
/exporter-logs-otlp-proto) failed the image build. That change needs a full
`next build` as its gate, which makes it its own PR rather than a rider on the
bridge.
The bridge does not depend on it. It resolves the Logs API lazily, per emit, so
a second bundled module copy is survivable rather than fatal, and
skipped{reason="no_provider"} is the runtime signal if one ever shows up. The
list is now byte-identical to main; only a comment recording this is added.
* fix(telemetry): register the OTel sink from the server entry, not the shared shim
`src/server/logging/client.ts` is in the CLIENT bundle. It reads as server-only
and is not: _app.tsx -> server/services/system-cache -> server/redis/fail-open-log
reaches it, so a module-scope import there is bundled for the browser too.
Importing the bridge from it pulled prom-client into the browser graph and failed
the image build with `Can't resolve 'cluster' / 'fs' / 'v8'`. Nothing local caught
it: typecheck, eslint, and both vitest projects were green, because only the
bundler walks that edge.
The sink is now REGISTERED rather than imported — instrumentation.node.ts, which
only ever runs on the server, calls setStructuredLogSink(emitOtelLog) at boot. The
shim keeps a type-only import, which is erased and adds no runtime edge.
This works because deps.emitLog is read per call rather than captured at
construction. That was incidental before and is now load-bearing, so a test pins
it: registering a sink AFTER the logger is built must still fire. Mutating the
factory to capture at construction fails that test and only that test.
|
||
|
briant
|
29ce14c96a |
Merge remote-tracking branch 'origin/main' into moderator-app-pages
# Conflicts: # packages/civitai-db-schema/prisma/schema.full.prisma # packages/civitai-db-schema/src/models.ts # pnpm-lock.yaml # src/pages/moderator/image-rating-review.tsx # src/pages/moderator/image-tags.tsx # src/pages/moderator/reports.tsx # src/server/services/cosmetic.service.ts |
||
|
Justin Maier
|
d1b3241814 |
chore(deps): upgrade to Next 16.3.0 and stop paying nested-async chunking in dev (#3742)
* perf(dev): stop paying server nested-async chunking in dev The flag exists to shrink emitted server chunks in a production build. A dev server never emits those chunks, so the walk over dynamic-import paths is pure cost there, on a module graph `_app` already makes large. Keeps the build-time behaviour from #3458 unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(deps): upgrade next to 16.3.0 16.3.0 is the first release carrying vercel/next.js#93788, which fixes a lock-order inversion in turbo-tasks-backend and removes an incorrect `unsafe impl Send` on dash_map_multi::RefMut that let a caller hold a shard write guard across an await. The symptom is a dev server that pegs most of the cores with flat memory and never finishes compiling a route, recoverable only by restart. Verified by reading the file at each tag: the unsound impl is present through 16.2.12, the last 16.2 release, and gone in 16.3.0. It was never backported, so no 16.2.x can pick it up. The `get_multiple_mut` equal-keys assert still exists in 16.3.0, so this is not expected to silence that panic — only to stop a panicking worker from wedging every other worker behind it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: commit the agent-rules block next dev writes into CLAUDE.md `next dev` on 16.3.0 appends a `nextjs-agent-rules` block to CLAUDE.md and re-adds it every run (next/dist/server/lib/generate-agent-files.js), so the choice is to commit it or carry an uncommitted change forever. It earns its place here: it points at `node_modules/next/dist/docs/`, which is already the authority this repo cites for Turbopack defaults, and the warning is aimed squarely at a pages-router app on Next 16. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Justin Maier
|
a86a39e481 |
chore(dev-server): spoke apps, branch-switch survival, and per-worktree env (#3741)
* feat(dev-server): run the SvelteKit spoke apps as daemon sidecars The apps under apps/ each need `vite dev` in their own directory so vite picks up that app's .env, plus readiness parsing, crash handling and a log buffer. AuthHub already encodes all of that, so a spoke is an AuthHub with a label. Ports are fixed per app rather than auto-assigned so a redirect between two of them (moderator -> auth) always lands in the same place, and --strictPort fails loudly on a collision instead of drifting to the next free port. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(dev-server): keep the dev server alive across branch switches Switching branches in place meant deleting .next and reinstalling, or the server would hang. The daemon now watches HEAD and handles the switch. It does NOT kill the dev server to do it. Measured on this repo, same routes, same machine: cold start after a kill ......... /models 42.6s left running across a switch .... /models 7.7-9.2s, /images 1.0-1.6s The running process keeps its in-memory module graph, so only what actually changed recompiles, and routes it isn't asked for stay warm. A restart is forced only when pnpm-lock.yaml or schema.full.prisma changes, since node_modules and the generated Prisma client can't be swapped under a live process. KILL_ON_BRANCH_SWITCH=true restores the old behaviour. Also adds: - PREWARM_ROUTES, compiled in the background on start and after each switch, so the ~45s first-route compile lands on the daemon rather than on your next click. Detail pages are separate routes from their list pages and need their own entry. - PER_BRANCH_DIST_DIR (default off) via distDir in next.config.mjs, with LRU eviction against both a count and a size budget. Off because a shared .next lets unchanged modules stay valid across branches, and each per-branch dir grows to ~4GB. - TUI session switching (s/Tab) plus a session counter, for running several worktrees side by side. - defender-exclusions.ps1: Windows real-time scanning over ~210k node_modules files is a large chunk of the cost on this machine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(dev-server): start a session from its own worktree's .env The env path fell back to the daemon's project root, so every session in every worktree ran on whichever tree happened to launch the daemon. A branch that adds a flag or points at a different database silently ran without it, and a NODE_OPTIONS set in a worktree .env never reached the process at all. Prefer the session worktree's .env, falling back to the project root only when the worktree has none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(dev-server): stop orphaning spokes, and say which .env a session loaded Review findings on this branch. Spoke apps were started but never stopped: /shutdown, SIGINT and SIGTERM each tore down sessions, the rgb proxy and the auth hub, and left the spokes running. Since spokes bind with --strictPort, one survivor makes the next start of that app fail EADDRINUSE against a process the daemon no longer tracks. The env-path change in 04114ab9 picks between two files that are not merged, so it can point a session at a different database with no receipt. Log the chosen path at start and return it as `envPath` in session status. Docs corrected where they described behaviour the code does not have — the worktree/root .env precedence, PREWARM_ROUTES having a default (it is empty, and prewarm-on-start needs HEALTH_CHECK_URL), and skill .env edits reaching a session that a branch switch deliberately does not restart. .env.example gains the ten knobs it was missing, and defender-exclusions.ps1 takes -ReposRoot instead of hardcoding one machine's layout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
briant
|
c451c702f1 |
Merge origin/main into moderator-app-pages (round 2)
Main advanced 11 commits after the previous merge, re-conflicting the PR. Brings it current again. Conflicts, same migration-seam pattern plus the db-queries PR (#3575) main just merged: - cosmetic.schema.ts / cosmetic.router.ts / cosmetic.service.ts: kept the branch's removal of grantCosmeticsToUsers/revokeCosmeticsFromUsers (moderator cosmetics grant -> spoke) and kept main's NEW stickers feature (getStickerCosmetics + getStickerCosmeticsSchema). Reconciled imports to the auto-merged router body, which uses only the sticker path. - package.json: kept all three workspace deps (@civitai/db-queries + @civitai/db-schema from main, @civitai/moderation from the branch). - pnpm-lock.yaml: took main's and reconciled via `pnpm install`. db-schema regeneration: main's db-queries PR restructured @civitai/db-schema. Ran db:generate (slim-schema + prisma generate) to make the generated Kysely types consistent, and dropped a stale `BaseModelLicensingFee` entry from updated-at-tables.ts (the model was removed on main; that generated file is stale on main too and its CI regenerates it). Verified: main app `tsc --noEmit` exit 0; spoke `svelte-check` 0 errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
briant
|
86c77399e1 |
Merge origin/main into moderator-app-pages
Brings the branch current with main (491 commits). Three code conflicts, all in the moderator-migration seam where main kept developing handlers the branch moved to the spoke: - article.router.ts / article.service.ts: kept the branch's removal of the article rating-review MODERATOR handlers (getArticleRatingReviews, getArticleRatingReviewCounts, resolveArticleRatingReview — now spoke-owned), and kept main's NEW article-image-scan feature (resolveArticleImageScan, rescanArticleImage + their schemas). The auto-merged router body already used only the image-scan procedures, so the imports were reconciled to match. - routers/moderator/index.ts: kept main's new imports (getModelChangeHistory, getImagesModRules) for its added moderator procedures. Two semantic fixes the auto-merge left dangling (main added consumers of things the branch had removed as "orphaned"): - Restored image.service `resolveIngestionError` — main's new resolveArticleImageScan reuses it. getIngestionErrorImages stays removed (still spoke-only, no caller). - Restored the `moderatorProcedure` import in article.router.ts for main's new resolveImageScan procedure. Verified: main app `tsc --noEmit` exit 0; spoke `svelte-check` 0 errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Briant Diehl
|
62a00997ea | Merge branch 'main' into feat/db-queries-kysely | ||
|
Zachary Lowden
|
0801071370 |
perf(build): enable Turbopack server-side nested async chunking (-316.7 MiB of server chunk source) (#3458)
* perf(build): enable Turbopack server-side nested async chunking Next's own default table has `turbopackClientSideNestedAsyncChunking` defaulting to TRUE in build mode, but `turbopackServerSideNestedAsyncChunking` defaulting to FALSE in both dev AND build. The server build therefore never received the async-chunk dedup the client build already has, so every async chunk group re-emits its entire module graph. Measured on the production server build: emitted chunk files 21,247 chunk source bytes 587 MB distinct modules 7,844 module copies emitted 1,176,346 (~150 copies per module) duplicated 535.8 MB (96.7% of chunk bytes) Enabling the flag on the same commit: chunk files 21,247 -> 5,830 (-72.6%) chunk bytes 587.4 MB -> 255.3 MB (-316.7 MiB, -56.5%) page entries 548 -> 548 (unchanged) build wall time 228s -> 312s (+37%) Runtime risk is low: the Turbopack runtime loader installs a module factory only `if (!moduleFactories.has(id))`, so duplicate copies are already registered-then-discarded today. This changes what is emitted, not module identity - no singleton or `instanceof` semantics change. Eager chunk sets are byte-identical before and after, so process startup work is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(build): trim the chunking comment to the mechanism, drop unverified aggregates An adversarial audit could not reproduce three of the figures baked into the config comment (7,844 distinct modules / 1,176,346 module copies / ~150 copies per module), and the "96.7% of chunk bytes duplicated" claim does not cohere with the other numbers in the same block: 535.8 MB is 91.2% of 587.4 MB, and 96.7% only holds against module-source bytes rather than chunk bytes. It also measured the real cost far lower than the comment claimed. The scoping pass's "+37% build wall time" compared against an anomalously fast local baseline. The actual CI build of this commit took 312.8s against a ~290s baseline median (+7.9%); a cold same-commit local A/B gave +13% wall / +19% compile. Keeping the mechanism and the shape of the trade in the config, and moving the measurement table to the PR, where it can be corrected without a code change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
briant
|
01ad026c66 |
Merge origin/main into moderator-app-pages
Brings the branch up to date with main (321 commits), incl. the canonical @civitai/shared package. Conflict resolutions: - @civitai/shared: took main's canonical package (browsing-levels + flags + index); the spoke keeps its allBrowsingLevelsFlag import (now main's Blocked-excluding semantics), per instruction. - next.config.mjs: serverExternalPackages took main's (adds @pyroscope/nodejs + @datadog/pprof); the auto-merged transpilePackages keeps both sides. - image.controller.ts + routers/moderator/index.ts: kept main's added imports. - pnpm-lock.yaml: regenerated via pnpm install against the merged package.json. Verified: main app `tsc --noEmit` exit 0; spoke `svelte-check` 0 errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
briant
|
43b75f54d9 |
feat(shared): add @civitai/shared package (Flags + browsing-levels)
New client-safe, dependency-free package for cross-app pure constants + utilities, wired into the main app (transpilePackages + workspace dep). First shared-package home for these primitives; app-level copies can adopt it to dedup later. - flags.ts: the Flags bitwise utility (port of the main app's ~/shared/utils/flags). - browsing-levels.ts: NsfwLevel bitwise levels + the generic browsing-level toolkit ported from ~/shared/constants/browsingLevel.constants — level groupings + flags, parseBitwiseBrowsingLevel/flagifyBrowsingLevel, labels + descriptions, severity/label helpers, and the browsing-level predicates. The App-Blocks/orchestrator/deprecated/UI/ reasons pieces stay app-specific. - allBrowsingLevelsFlag matches the main app (EXCLUDES Blocked); a distinct allBrowsingLevelsWithBlockedFlag covers moderator "show Blocked" contexts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
briant
|
b96b7c7772 |
feat(db-queries): Phase-0 @civitai/db-queries foundation + reports port
Scaffold the shared Kysely data-access package and wire it into the main app, per docs/db-queries-kysely-plan.md. - @civitai/db-queries: globalThis-backed client vars + connect(), nested-read (jsonArrayFrom/jsonObjectFrom) and toJson write helpers, and the reports queries (setReportStatus / setReportStatusMany) as the reference module. - Main app: kyselyDb.ts builds clients over the existing pgDb/datapacket pools and calls connect() at boot (instrumentation.node.ts); getKyselyWithoutLag twin of getDbWithoutLag; bulkSetReportStatus repointed to setReportStatusMany. - Testing: offline compile-only harness (Kysely DummyDriver) + vitest config, reports compiled-SQL tests (incl. the empty-array IN () guard), and a two-tier testing section in the package README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
briant
|
08bbfeaca5 |
chore(monorepo): add @civitai/moderation + wire buzz/email/notifications into the spoke
Foundational wiring for the moderator image-moderation slice.
- @civitai/moderation (NEW package): the cross-app moderator-action contract —
action-input zod schemas + MOD_ACTION names (schema.ts) and a no-retry
createModeratorClient (client.ts) the main app uses to call the spoke's
/api/mod/*. Producer (spoke endpoint) and consumer (main-app client) share the
same schemas so they can't drift (same pattern as @civitai/notifications).
- Declare @civitai/{moderation,buzz,email,notifications} for the spoke
(apps/moderator/package.json) + add them to vite ssr.noExternal so the raw-TS
packages transpile under the SvelteKit build.
- Add @civitai/moderation to next.config transpilePackages (+ notifications,
already imported by the spoke's tos-violation notify).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Zachary Lowden
|
d0a795c1d2 |
feat(profiling): Grafana Pyroscope continuous CPU profiling SDK (dark by default) (#3112)
* feat(profiling): add Grafana Pyroscope continuous CPU profiling SDK (dark by default) Adds the @pyroscope/nodejs in-process wall/CPU profiler behind a PYROSCOPE_ENABLED env gate (default OFF). Ships as a complete no-op: the SDK is dynamically imported ONLY when the flag is armed, so an unset flag never loads the package or its native @datadog/pprof addon. When armed it runs a 100Hz wall+CPU sampler on a separate thread and pushes pprof to a configurable ingest endpoint (PYROSCOPE_SERVER_ADDRESS), tagged by pod/version/pool for diff-by-version flamegraphs. Purpose: continuous server-CPU profiling to rank and book per-request CPU optimizations (the live lever for our peak-capacity-capped web tier). - src/server/pyroscope.ts (new): flag-gated dynamic init, mirrors the cpu-profiler.ts arm-time-never-crashes discipline. Wall/CPU only (no heap), collectCpuTime on. - src/instrumentation.node.ts: fire-and-forget registration by the existing profiler hooks. - next.config.mjs: externalize the SDK + @datadog/pprof (native addon) from the server bundle. - package.json / pnpm-lock.yaml: add @pyroscope/nodejs@0.6.1 (exact pin — fresh native package). - .nvmrc: 20.13 -> 20.20.2 (the SDK's node engines floor). No behavior change with the flag unset (how it ships). * harden(pyroscope): make the profile flush interval env-tunable The flush (serialize -> protobuf encode -> push) runs on the MAIN thread every flushIntervalMs (default 60s), so on a CPU-ceiling-capped pool it's a periodic on-loop blip at peak. Wire PYROSCOPE_FLUSH_INTERVAL_MS so a Stage-2 canary can lengthen it from env (no code change) if event-loop-lag p99 shows the stall is material. Still dark by default; unset = SDK default 60s. (Addresses the sole arming-time risk from the #3112 adversarial audit.) |
||
|
Briant Diehl
|
9e27404a9a |
Merge pull request #2892 from civitai/feat/notifications-monorepo-migration
Feat/notifications monorepo migration |
||
|
briant
|
0bea99cf84 |
feat(buzz): extract @civitai/buzz server-side client; refactor buzz.service onto it
- New @civitai/buzz package: server-only buzz-service client (fetch transport +
typed per-endpoint methods + wire response types) plus the browser-safe
account-type name-map. BuzzTypes in the app is now a facade delegating to it.
- Refactor src/server/services/buzz.service.ts onto the client; behavior-preserving
(same request bodies, retries, and tRPC error mapping via mapError).
- responses.ts audited field-by-field against the .NET buzz service (source of
truth); fixes real wire-type errors, incl. CreateTransactionsResponse
({ transactions: string[]; conflicts: string[] }) and CreateTransactionResponse
(+remainingBalance).
- createBuzzTransactionMany: reconcile expected vs succeeded+conflicted and log
buzz-transaction-many-dropped when the batch silently drops transactions (e.g.
insufficientFunds), which the endpoint reports in neither array.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
briant
|
927e31bfee |
feat(notifications): fold notification domain into apps/notifications + @civitai/notifications
Move the notification domain out of the monolith (and the external notification-server repo) into an in-repo app + shared package, and cut the monolith off the notification DB entirely. - @civitai/notifications (package): the stable producer/reader seam — zod schema contracts, NotificationCategory/signal constants (now the single source of truth; src/server/common/enums re-exports it), and an HTTP client (create/bulk/query/count/markRead/exists/cleanup) that POSTs to the app. - apps/notifications (app): Fastify producer + read API (shared-secret auth, internal ingress) and the fan-out poll worker, on the shared @civitai/* clients. Owns the settings-filter, base-row read, unread-count cache, replication-lag routing, and mark-read retry/queue. - Monolith cutover: notification.service, send-notifications, club.service, reaction.controller, and the admin cleanup endpoint call the package directly; deleted notifDb.ts, the notif prom gauges, getNotifDbWithoutLag, and the dead notification-cache; getClient dropped its notification instances; NOTIFICATION_DB_URL/_REPLICA_URL made optional. - @civitai/db: extracted generic createPool/createClients (getClient now wraps them) and a redis-agnostic createLagTracker (monolith db-lag-helpers + the app both build on it). Verified: monolith + app typecheck, 28 unit tests, two code reviews, and a live smoke of the producer/read path scoped to user 5. NOT a merge-and-deploy — this is a hard cutover; follow docs/plans/notifications-predeploy-checklist.md (app must be deployed with the worker gated off and NOTIFICATIONS_ENDPOINT set before the monolith ships). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Briant Diehl
|
f905920d5f |
feat(auth): centralized-hub login cutover — remove in-app login UI
Login moves fully to the centralized auth hub (auth.civitai.com); the main app no longer renders any login UI of its own. - Remove the in-page login UI + social-login components (LoginContent, LoginModal, EmailLogin, SignInError, SocialButton/Social/SocialLabel) and migrate every consumer (AccountsCard, IframeHost, /login, discord/link-role) to the hub popup / full-page redirect. /login is now a pure server-side hub redirect; AccountsCard is self-contained (local provider map). - Hub owns its login analytics: emit LoginRedirect to the shared ClickHouse tracker; thread the login `reason` end-to-end (popup + full-page) and re-home referral attribution off the deleted LoginContent cookie via the post-login URL. - Hub login hardening: Discord role_connections.write scope + granted-scope storage on the Account; email magic-link grace window (not single-use-deleted, so link scanners can't lock out the real click); plus-address signup block; never store provider avatars; prompt=select_account passthrough; default returnUrl; fail (not switch accounts) on a lost-session link callback. - @civitai/auth hygiene: drop single-consumer OAUTH_PROVIDERS/OAuthProvider (keep ProviderId + hubLoginUrl). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Briant Diehl
|
307824d106 |
feat(auth): impersonation wiring + browser-client adoption + read-path revocation
- Impersonation goes through AccountProvider.impersonate/exitImpersonation → authClient → /api/auth/impersonate (POST/DELETE proxy); ImpersonateButton driven by session.impersonatedBy; removed the ogAccount localStorage. - AccountProvider + components call @civitai/auth/client (no inline auth fetch); reload-in-place on impersonate. - sessionClient injects isRevoked (reject revoked tokens on cache hit); sync.ts no longer leaks the raw error. - Readability: collapsed identical logout/logoutAll; flattened swapAccount to guard clauses; centralized the three JWT decoders into token-claims.ts; deduped the CookieWritable interface; shared civ-cookie helper. - next.config: transpile @civitai/auth (needed for the browser client bundle). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Briant Diehl
|
552a53e27c |
Merge origin/main into monorepo-bootstrap
Brings in main's App Blocks feature (+ @civitai/app-sdk, @civitai/blocks-react). Conflicts resolved: - package.json: keep @civitai/auth (workspace) + main's @civitai/app-sdk + @civitai/blocks-react - .github/workflows/pr-check.yml: accept main's deletion - src/shared/utils/prisma/models.ts: keep our @civitai/db-schema re-export shim - db-helpers.ts / redis/client.ts / prom/client.ts: keep our package-extracted shims (ours) - oauth/authorize.ts: keep both imports (OIDC nonce + app-block client check) - pnpm-lock.yaml: reconciled via pnpm install Ported main's App Blocks infra additions INTO our extracted packages so the incoming code compiles against the package architecture: - @civitai/redis: REDIS_KEYS.BLOCKS + REDIS_SYS_KEYS.BLOCKS keys; withRedisCommandTimeout (app shim) - @civitai/db: 'apps' pool instance + APPS_DATABASE_URL/appsUrl - @civitai/telemetry: App Blocks metrics (blockBuzzAttribution, appStorageOps/QuotaExceeded/Latency) Typecheck: 0 errors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Zachary Lowden
|
b5c7bfb8b3 |
fix(og): ship @vercel/og in standalone so /api/og stops 500ing (#2509)
* fix(og): ship @vercel/og in standalone so /api/og stops 500ing /api/og renders via next/og's `ImageResponse`, which on the nodejs runtime lazily require()s `next/dist/compiled/@vercel/og/index.node.js` (+ its resvg/yoga WASM + fonts). @vercel/nft can't follow that dynamic require, so with output:'standalone' the file is traced OUT of the image — every origin (cache-miss) /api/og render then throws `Cannot find module ... index.node.js` → res.status(500). Impact: the dominant app-emitted 500 source on dp-prod (~1.9/s, ~74% of the app-500 floor) since the Next 16.2.7 upgrade. Cloudflare edge-caching of OG images masked it for popular entities, but cache-bypass returns 500 for every type (verified: model/image/post/article). Degrades social-share/OG previews for any uncached entity. Fix: add `/api/og` to outputFileTracingIncludes with the compiled @vercel/og dir (entry + WASM + fonts). Two version-agnostic globs (no hardcoded next@hash) to be robust to pnpm symlink resolution. Verification: requires a standalone build — confirm on the PR preview that a cache-busted /api/og?type=model&id=<id> returns 200 (currently 500 at origin). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(og): give FallbackCard logo explicit width so the fallback can't 500 Verifying the standalone-packaging fix on a preview exposed a SECOND, pre- existing /api/og 500 that the module-not-found error had masked: the catch-path `FallbackCard` renders the wordmark logo as `<img height={60}>` with NO width. satori then throws "Image size cannot be determined. Please provide the width and height of the image." — so the fallback ImageResponse itself throws and the outer catch returns 500. This means ANY missing entity (e.g. a deleted/invalid id from a social crawler) or any OgCard render error hit FallbackCard and STILL 500'd, even with the module present. Verified on the pr-2509 preview: existing model/image → 200, but missing post/article/bounty/challenge → 500 with this exact error. Fix: logo_dark_mode.png is 142x30 (≈4.733), so set width={284} at height={60} to preserve aspect and satisfy satori. Now the fallback renders instead of throwing. (Other cards already set both dims; only FallbackCard omitted width.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Briant Diehl
|
be0ccbc27b |
Merge origin/main into monorepo-bootstrap
Conflicts resolved:
- package.json: keep @civitai/auth workspace dep + take @civitai/client 0.2.0-beta.71
- src/pages/login/index.tsx: take main's session?.user guard (login-loop fix)
- .github/workflows/pr-check.yml: keep our lint:packages step + main's updated
no-io-in-transaction comment
- src/shared/utils/prisma/{enums,models}.ts: keep our @civitai/db-schema re-export
shims; regenerated the package's generated enums/models from the merged schema
(CsamReportType gains ExternalLink)
- pnpm-lock.yaml: reconciled via pnpm install
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Justin Maier
|
21df8012cb |
chore(config): add civitai-dev RGB origins to allowedDevOrigins
Allow civitai-dev.green/blue/red as dev origins (RGB proxy) for everyone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Briant Diehl
|
ee89847977 |
Merge remote-tracking branch 'origin/main' into monorepo-bootstrap
# Conflicts: # .gitignore # src/server/prom/client.ts |
||
|
Zachary Lowden
|
96f484e46e |
feat(build): ship server source maps for prod CPU-profile de-minification (#2460)
* feat(build): ship server source maps for prod CPU-profile de-minification
Prod pods capture V8 .cpuprofiles to find event-loop blockers, but the
standalone image shipped zero server .js.map files, so frames were minified
and unnameable (e.g. `p @ src_17njnbr._.js:0`).
Build change (Turbopack / Next 16):
- Under Turbopack the only source-map lever is `turbopackSourceMaps`, whose
build-time default IS `productionBrowserSourceMaps` (already true). So server
chunk maps (.next/server/**/*.js.map) are already emitted at build time;
`experimental.serverSourceMaps` is webpack-only and ignored by Turbopack.
Reworded next.config.mjs comment to state this accurately.
- output:'standalone' traces via @vercel/nft, which follows import/require/fs
and does NOT copy sibling .map files, so the maps were dropped from the image.
Dockerfile now stages just the server-chunk maps (structure-preserving tar)
in the builder and overlays them onto .next/server in the runner.
Maps are inert at runtime (loaded only by an inspector/stack resolver) -> no
request-path perf cost. Cost is build time + ~order-of-tens-of-MB image size.
Resolver tool:
- scripts/resolve-cpuprofile.mjs maps each frame's (chunk.js, line, col) back to
original {source, line, name} via the `source-map` package, ranks hottest
self-time leaves, and reconstructs the longest-synchronous-block stack, named.
- Verified end-to-end against a real esbuild-minified bundle + map (synthetic
profile frames resolved back to original fn names + .ts locations). Prod proof
awaits the next map-enabled deploy + a fresh capture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(build): publish server maps as on-demand artifact, not in runtime image
The previous approach overlaid all server .js.map files into the runtime
image, adding ~761 MB to every prod pod (11,305 maps) for a debug aid that
is only needed OFFLINE by the cpuprofile resolver.
Instead:
- Dockerfile: revert the runtime-stage map COPY (runtime image is lean again).
Add a `FROM scratch AS maps` target holding ONLY the staged server maps; it
shares every builder layer (cache hit) and is published separately.
- resolve-cpuprofile.mjs: add `--image <tag-or-ref>` mode that fetches that
build's maps from ghcr.io/civitai/civitai-web-maps:<tag> via `crane export`
(falls back to `oras pull`), then resolves. Keeps the local `--maps <dir>`
mode. Maps are keyed by the exact image tag so a profile from image X
resolves against X's maps.
The Tekton build pipeline (datapacket-talos) publishes the `maps` target to
the sibling repo after the main build+push, reusing the same ghcr creds, as a
non-fatal step.
Verified end-to-end: pushed a synthetic maps image to a local registry, then
`--image` fetched + extracted it and de-minified frames to original src/*.ts
functions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(source-maps): correct stale next.config comment + clean up resolver temp dir on error
Audit follow-ups on the artifact-based source-maps PR:
- next.config.mjs: the comment still described the pre-revision behavior (maps
baked into the runtime image). Reworded to reflect the on-demand maps artifact.
- resolve-cpuprofile.mjs: wrap the post-fetch body in try/finally so the fetched
maps temp dir (hundreds of MB) is always cleaned up, even if resolution throws
(previously leaked on the --image error path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Briant Diehl
|
d572773c31 |
Merge branch 'main' into monorepo-bootstrap
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Briant Diehl
|
6aa4d37392 |
Merge origin/main into next-16-migration (2nd)
Conflicts resolved: - next.config.mjs: kept main's `transpilePackages: ['superjson']` (superjson v2 upgrade #2425) alongside our `serverExternalPackages` and top-level `outputFileTracingIncludes`. - pnpm-lock.yaml: regenerated via `pnpm install` against the merged package.json (next ^16.2.7 preserved; superjson v2). - src/pages/api/trpc/[trpc].ts: auto-merged — kept our Next 16 withAxiom typing and main's new TOO_MANY_REQUESTS auth-skip. - src/middleware.ts: main's edits were functionally identical to our renamed src/proxy.ts; rename-aware merge kept proxy.ts (no duplicate). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Zachary Lowden
|
92eaff16e4 |
perf(trpc): upgrade superjson 1.9.1 → 2.2.6 (#2425)
Replaces the closed devalue migration (#2410) with the lower-risk lever from the best-practice audit. We were 3 major versions behind (1.9.1 → 2.2.6, the latter actively maintained), and every benchmark that motivated the devalue swap compared devalue only against superjson 1.x — the stale version we run. superjson 2.x is the era after its serialization-perf work, so this directly tests "is the serializer the cost?" with none of devalue's downsides. Why this over devalue: - LENIENT, like 1.x — still coerces Prisma Decimal / dayjs / SDK objects / Error / class instances instead of throwing. No per-route 500-risk tax, no SSR full-page-500 surface (the recurring problem with devalue). - Drop-in: identical `{json, meta, v:1}` wire format (verified) → cross-version compatible during the rolling deploy, and the existing transformer wiring (`superjson.serialize`/`.deserialize`, default import at 3 sites) is unchanged. - ESM-only is superjson 2.x's only breaking change (Node ≥16) — handled by adding `superjson` to `transpilePackages` for the standalone server bundle (same mechanism devalue needed). No `require()` of superjson exists in src/. Verified v2: default export intact, serialize/deserialize round-trip Date/Map. Transitive deps copy-anything 3→4, is-what 4→5 are superjson's own. The higher-ceiling fix (epoch dates + no transformer on the feed path) remains a separate follow-up. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Briant Diehl
|
9e233a3270 |
Merge origin/main into next-16-migration
Conflicts resolved: - pnpm-lock.yaml: regenerated via `pnpm install` against the merged package.json (kept next ^16.2.7). - src/pages/api/trpc/[trpc].ts: kept main's native `allowMethodOverride: true` handler (+ responseMeta caching, onError Axiom logging) and applied the Next 16 withAxiom typing fix to the export; dropped the branch's now-redundant manual restoreMethodOverride(). - next.config.mjs: auto-merged, then moved main's `outputFileTracingIncludes` from `experimental` to top-level (it moved to top-level in Next 15; under `experimental` on Next 16 it is silently ignored, which would regress the src/static-content standalone tracing fix). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Briant Diehl
|
3530a85f30 |
Migrate to Next.js 16 + Turbopack; fix Turbopack-dev runtime regressions
Build/deps/config:
- next, eslint-config-next, @next/bundle-analyzer → ^16; Turbopack is the
default build bundler. Added turbopack:{} config, dropped removed config keys.
- Removed all 151 typed-scss-modules *.module.scss.d.ts in favor of ambient
declarations in src/types/css-modules.d.ts (Turbopack panics on the .d.ts).
- CSS modules: rewrote bare :global to function form for Turbopack.
- trpc/[trpc] handler typing; bundler-agnostic applySourceMaps.
Runtime fixes (Turbopack dev/prod):
- CAConsentManager: static-import instead of next/dynamic. A dynamic component
wrapping the whole app async-loads its chunk under Turbopack dev, causing a
whole-tree hydration mismatch that re-mounts the app and orphans the server
DOM (the "double layout"; US:CA-gated). General rule: wrapper dynamics remount,
leaf dynamics are fine.
- ReactQueryDevtools: load client-only (ssr:false) — not SSR-safe under Turbopack.
- SharedWorkers (signals + civitai-link): Turbopack doesn't compile .ts
SharedWorker entries (vercel/next.js#74842), breaking dev AND the prod build.
Pre-bundle each with esbuild to public/workers/*.js (scripts/build-workers.mjs,
wired into predev/prebuild) and instantiate via static path.
See docs/next-16-migration.md for full details and remaining open items.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Justin Maier
|
4d9ec59205 |
fix(content): drop duplicate outputFileTracingIncludes
Briant landed the same standalone static-content fix in |
||
|
Justin Maier
|
d48a46b0af |
fix(content): bundle src/static-content into standalone output
The content endpoints (content.get / checkTosUpdate tRPC procedures and the
/content/[[...slug]] and /safety pages) read markdown from src/static-content/*.md
at runtime via fs (resolve('src/static-content/...')). Next output-file-tracing
cannot follow a dynamically-built fs read, so the .md files were only ever
included in standalone builds incidentally.
A lockfile regen (
|
||
|
Briant Diehl
|
bb605de707 |
@
fix(build): ship src/static-content markdown in standalone output Several entry points read markdown from src/static-content at runtime via fs (dynamic string paths @vercel/nft cannot trace). With output: standalone the build only ships traced files, so the markdown was missing from the deployed image and every read hit ENOENT -> 500/404 (/safety, /content, and the site-wide checkTosUpdate TOS check). Worked locally only because the full source tree is present. Add experimental.outputFileTracingIncludes keyed by every read site so the files are force-included into the standalone bundle. On Next 14 this key lives under experimental (top-level only takes effect in Next 15). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> @ |