Files
civitai__civitai/docs/frontend-perf-audit-2026-04.md
T
Zachary Lowden c25f555b62 security(repo): remove production data, internal topology, and open-findings docs from this public repo (#3670)
This repository is public and permanently world-readable, including history and
the docs/ and .claude/ trees. An audit of the recent merge window plus the
standing always-public surface found several classes of content written with an
internal-audience mindset. This removes or genericizes them and adds a standing
rule to CLAUDE.md so the class does not recur.

Removed — production data about real people:
- Two spent admin/temp backfill scripts with real user identifiers and account
  attributes inlined as literal arrays (435 and 189 entries). Both were one-shot
  migrations that already ran, with zero external references.
- An internal triage doc reproducing named creators' private financial detail
  and message contents, and two moderation shortlists containing named creators
  with internal content classifications.
- A tracked sample payload under local/ (a path that is otherwise gitignored)
  carrying real workflow telemetry and a staging webhook token.

Removed — security reviews listing unfixed findings:
- Five auth review and cutover documents that enumerated OPEN findings with
  file:line pointers. A findings list on a public repo is a to-do list for an
  attacker. The records themselves are valuable and belong in the private repo.
- Dangling references to them were rewritten rather than left broken; several of
  those reference lines restated the findings inline, so the link text was part
  of the disclosure.

Removed — operational runbooks and vendored foreign manifests:
- A production pod-exec/profiling runbook including container reconnaissance.
- Three deploy workflow files copied from other repositories, carrying an
  internal deploy-webhook host and serving no function here.

Genericized rather than removed, because the docs are otherwise useful:
- The postgres-query skill no longer documents the bastion path to production
  or which private-repo files hold the SSH keys and database password; it now
  points at an infra owner.
- The flipt skill no longer publishes the endpoint hostname or the authentication
  posture of the service behind it.
- The deploy-status skill no longer publishes the canary rollback thresholds,
  which described the conditions required to force a production rollback.
- Blanked a filled-in bucket name and endpoint in one .env.example that its
  sibling app correctly left empty; genericized a real analytics org id.
- Replaced private infra-repo directory paths in two .env.example headers, a
  README, a service comment, and a one-off script.
- Replaced developer home directory paths across seven docs.
- Dropped a token-hash construction and a "known gap" admission from the
  post-deploy checklist.

CLAUDE.md gains a Security section stating plainly that the repo is public and
listing the eight content classes that belong in the private infra repo, with a
before-committing test and the note that removal is not remediation.

Source changes in this commit are comment-only. The two deleted route files had
no importers.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 23:51:10 -05:00

14 KiB
Raw Blame History

Frontend Performance Audit — April 2026

Heap snapshot + Performance profile analysis of civitai.com during a brief feed scroll session. Captured 2026-04-22 by Justin (logged out, Chrome). Analyzed by Claude Code by parsing the raw snapshot/trace JSON (see .claude/worktrees/perf-heap-fixes/docs/ and the scripts in C:\tmp\perf-analysis if you want to re-run).

Caveat. The capture was logged out — no websocket signals, no live metrics, no chat, no notifications. Every number below is a floor, not a ceiling. A logged-in capture will amplify the store/observer/animation paths.

Branch offset. main (what civitai.com runs) does NOT yet include 343778e1a ModelCard optimizations or the other unreleased work on this branch. Some fixes described below are already landed in this branch ahead of main — see "Status" markers. Relatedly, there is a separate feed-card-dom-audit.md by Briant covering DOM reduction + render cost at feed scale; the two audits are complementary.


TL;DR

  • +29MB heap after a brief scroll (61MB → 90MB fresh → scroll).
  • 26,839 detached DOM nodes (was 3). Real leak, not just churn.
  • 3,920 150-ms setTimeouts installed in 102s (38/sec) from one app chunk — a debouncer is being reset every scroll tick.
  • One requestAnimationFrame loop runs at 60Hz the whole time the feed is open, driving Layerize (1,627×), Paint (4,189×).
  • Feed cards use the @number-flow/react library (AnimatedCount / LiveMetric) for every metric. Each instance spawns a custom element + ShadowRoot + ~10 digit spans + 4-6 CSSStyleDeclarations.
  • A PerformanceObserver is running with buffered: true and never clears — PerformanceEventTiming entries leak linearly.

Heap diff: fresh → brief feed scroll

Metric Fresh After scroll Δ
Total self size 61.3 MB 90.4 MB +29.1 MB
Node count 1,385,045 2,179,815 +794,770
Detached DOM 3 26,839 +26,836 (+3.5 MB)

Top size-growth constructors

Constructor Δ count Δ size
object::Object +174,919 +5.24 MB
array::(object elements) +48,487 +2.14 MB
object::uW (minified) +16,131 +1.61 MB
array::(object properties) +10,526 +1.54 MB
closure:: +46,035 +1.32 MB
object::system / Context +42,691 +1.24 MB
native::Text +13,558 +1.23 MB
native::CSSStyleDeclaration +19,365 +1.13 MB
native::SVGPathElement +3,091 +0.52 MB
native::V8EventListener +12,358 +0.49 MB
native::DOMTokenList +7,077 +0.41 MB
native::PerformanceEventTiming +1,381 +0.36 MB

AnimatedCount / NumberFlow footprint

During the same scroll:

  • ShadowRoot 249 → 1,164 (+915)
  • native::<span class="digit__num" inert style="--n: 0..9"> — each digit added ~1,1001,500 instances. Across 09 that is ~10k digit spans.
  • native::<span class="AnimatedCount_wrapper__mdpqx"> 245 → 1,150 (+905)
  • native::SVGAnimatedLength/Transform/String/Number/Rect/PreserveAspectRatio — each grew by ~4,100 (SVG icons inside NumberFlow shadow DOM + Tabler icons on cards)

The library is cool. It is extremely expensive at card-grid scale.


CPU profile (102s trace)

Metric Value
Sum of X events 102.1 s
Long tasks (≥50 ms) 91
Heaviest 6 tasks 180225 ms each (React commits, chunk 30548-*.js)
scroll events dispatched 560 (avg 4.3 ms each, total 2.4 s)
rAF service calls (PageAnimator::serviceScriptedAnimations) 1,627
Layerize 1,627× (7.9 s total)
Paint 4,189× (1.54 s total)
IntersectionObserver::computeIntersections 3,316 calls

Timer churn

Top TimerInstall callsites (102 s window):

Count Timeout Source
3,920 150 ms civitai.com/_next/static/chunks/30548-8821e4834a2871a8.js:4
559 300 ms civitai.com/_next/static/chunks/pages/_app-4a3afcd97c94cf70.js:11
68 100 ms _app-*.js:11
41 500 / 5000 ms Snigel ad engine
34 34 requestAnimationFrame registrations also bound to _app-*.js:11

4,718 timers installed, 479 fired, 10,344 removed. The ratio screams debouncer reset storm: almost every install gets cancelled and replaced.

EventDispatch breakdown

Type Total Count Avg
scroll 2,401 ms 560 4.3 ms
wheel 36 ms 221 0.16 ms
pointermove 31 ms 260 0.12 ms
load 21 ms 292 0.07 ms

Scroll handlers are doing 4 ms of JS per event. That is where the 150 ms debouncer is getting reset 38 times per second.


Root causes — what's actually going on

1. NumberFlow in every card is a DOM bomb

src/components/Metrics/AnimatedCount.tsx wraps @number-flow/react, which is a web component with shadow DOM and per-digit rAF animation. It is used via LiveMetric across 16 components, including ArticleCard, ModelCard, ImagesCard, CreatorCardSimple, UserStatBadges, Reactions, ModelVersionDetails. Each card renders 47 metrics. A 50-card feed = 200350 NumberFlow instances = 200350 shadow roots, thousands of digit spans, a permanent 60Hz rAF loop.

This is the #1 contributor to the ShadowRoot, digit__num, CSSStyleDeclaration, Layerize, and Paint growth.

2. Per-card IntersectionObserver instead of shared

Status (resolved): MetricSubscriptionProvider has been removed entirely. Cards now share a single observer via the polymorphic ElementInView primitive, which is built on the shared IntersectionObserverProvider. Visibility-gated subscription is expressed via <Metrics useLive={inView === true}> rather than a per-card observer. See signal-refcount-known-issues.md and signal-topic-subscription-overhead.md.

src/components/Metrics/MetricSubscriptionProvider.tsx:77-90 creates a brand new IntersectionObserver for every card. The repo already has a shared one (src/components/IntersectionObserver/IntersectionObserverProvider.tsx) that multiplexes observations through a single observer. Using the shared provider here would cut computeIntersections by ~30×.

Equally important: per-card observers that aren't guaranteed to unsubscribe before the card unmounts are a likely source of the 26,839 detached nodes.

3. 150 ms scroll debouncer in chunk 30548

Every scroll tick (560 over 102 s) resets a 150 ms setTimeout. That chunk is a big aggregated app chunk; the exact module will need a source-map lookup or grep setTimeout.*150 with scroll-adjacent surrounding code. Candidates from the repo:

  • MasonryProvider + useBrowsingLevelDebounced recompute items in MasonryGridVirtual.tsx:6178 whenever the debounced browsing level fires.
  • createAdFeed re-slots ads.
  • Any Mantine useDebouncedValue(..., 150) in a scroll-adjacent component.

4. Stuck PerformanceObserver

PerformanceEventTiming +1,381 entries per short session, +0.36 MB/session. Somewhere we registered new PerformanceObserver({ type: 'event', buffered: true }) and never call takeRecords() or disconnect(). Classic observability leak — probably web-vitals instrumentation.

5. Tabler icon explosion

SVGPathElement +3,091, SVGSVGElement +1,040, 6 different SVGAnimated* growing +4,100 each. Every Tabler <IconX /> is an inline SVG tree with animated attributes. Feed cards inline 1525 icons each; no memoization, so they're re-created on every parent render. Mid-term fix: SVG sprite. Short term: React.memo the card.

6. Minified object::uW (+16k, +1.6 MB)

Most likely @tanstack/react-query's QueryObserver. Suggests unstable queryKey shapes (object literals created per render) or unbounded infinite query cache. Needs source-map lookup to confirm.


Fix plan (priority order)

P0

  1. Gate AnimatedCount to visible cards. Render a plain formatted number when offscreen; only instantiate NumberFlow when the card is intersecting. Halves DOM size and kills the always-on rAF loop. Status: done in this branch (perf(metrics): gate NumberFlow animation to visible cards).
  2. Switch MetricSubscriptionProvider to the shared IntersectionObserverProvider. One observer for the feed instead of one per card. Fixes computeIntersections churn and a detached-node vector. Status: superseded. MetricSubscriptionProvider was removed entirely. Subscription is now a hook (useMetricSubscription) gated by the polymorphic ElementInView primitive (built on the shared observer). Additionally: SignalProvider refcounts topics so duplicate subscribers dedupe at the hub, and the per-card 60s keep-alive timer was replaced with event-driven re-registration on reconnect. See signal-refcount-known-issues.md.
  3. Find and disconnect the stuck PerformanceObserver. Investigated — no PerformanceObserver registration in our source (rg "new PerformanceObserver" empty; no web-vitals / @sentry / @vercel/analytics / posthog deps). The +1,381 PerformanceEventTiming retention is coming from a third-party script, most likely Snigel's ad engine (adengine.snigelweb.com/.../adngin.js was the #2 TimerInstall source and regularly probes INP for bid quality) or a browser extension that was loaded during the capture (React DevTools + Metamask content scripts were present). Status: not in our code — no fix available from our side. If we add our own perf instrumentation later, be sure to disconnect observers on page unload.
  4. Track down the 150 ms scroll debouncer. Found it: @tanstack/virtual-core's scroll listener ends scrolling via an internal debounce(..., isScrollingResetDelay) (default 150 ms). The scroll handler calls fallback() on every scroll event, which resets the timer — reinstalling one setTimeout(150) per scroll tick. The three useVirtualizer callsites (MasonryGridVirtual, MasonryColumnsVirtual, pages/user/downloads.tsx) are the source of the 3,920 timer installs. Status: fixed in this branch — all three now pass useScrollendEvent: true, opting into the native scrollend event (Chrome 114+, Firefox 109+, Safari 18.2+). virtual-core falls back to the 150 ms debounce automatically on older browsers.

P1

  1. React.memo feed card components with stable prop references. Avoids re-rendering every card on unrelated store updates. Status: done in this branch for ArticleCard, BountyCard, CollectionCard, ChallengeCard, ComicCard, CreatorCardSimple. ModelCard/ImagesCard/PostsCard were already memoized.
  2. Memoize inline Tabler icons (or move to a sprite sheet). 1000 SVG trees per 50-card feed is cheap to kill. Status: deferred. With P1.5 in place icons only re-create on card mount (virtualizer scroll-in). True win requires a sprite sheet — not a shallow edit. Track separately if feed mount cost is still hot after these P0+P1 changes.
  3. Audit uW observer (queryKey stability). Either a source-map lookup or dev-build sanity check; look for { ... } object literals in queryKey. **Status: needs source-map — the minified uW name could be a Query, QueryObserver, or MutationObserver from tRPC/React Query. Worth running pnpm build --profile and inspecting the chunk, or spot-check hot hooks (useLiveMetric, useBuzzTippingStore, useReviewedModelIds) for queryKey stability.

P2

  1. Audit useSignalTopic cleanup when topic becomes undefined. Status: refactored. useSignalTopic now goes through registerTopic / releaseTopic on a refcounted provider; cleanup is explicit and tested in /testing/metrics-refcount. Topic-undefined transitions still need spot-checking after the refactor.
  2. Drop NumberFlow animation entirely for low-motion metrics (downloads, likes); use CSS pulse on value change instead.
  3. MasonryGridVirtual.tsx:61useBrowsingLevelDebounced + adsReallyAreEnabled cause items recompute every debounce fire. Guard with useMemo key stability.

Next capture (please)

  • Logged-in session, same pattern: fresh snapshot → 60 s feed scroll → second snapshot → 60 s idle → third snapshot.
  • Same 30-60 s Performance trace, "scroll feed" only (no navigation) for cleaner data.

Expected amplification when logged in:

  • Signal websocket + useMetricSignalsStore deltas → every card re-renders on every delta (even with useShallow, the MetricSubscriptionContext.Provider value is new each render — see point 1 of P1 above).
  • Notification + chat signalr loops → more rAFs and timers.
  • object::uW and detached DOM will likely explode further.

Reproducibility

Raw artifacts live in <local-path>/Downloads\perf\:

  • fresh.heapsnapshot (125 MB)
  • brief feed scroll.heapsnapshot (196 MB)
  • sitting for 5 minutes in background.heapsnapshot (163 MB, truncated — unusable)
  • Trace-20260422T191137.json (160 MB)

Analyzer scripts in C:\tmp\perf-analysis\:

  • analyze-heap.mjs — aggregates nodes by (type, constructor), writes <name>.summary.json.
  • diff.mjs — diffs two summary JSONs.
  • analyze-trace.mjs — event categories, long tasks, rendering pipeline.
  • find-timer.mjs — groups TimerFire by timerId, dumps TimerInstall stacks.
  • timer-stats.mjs — TimerInstall frequencies by callsite, rAF callsites, EventDispatch by type, install/fire/remove counts.

Run with node --max-old-space-size=12288 analyze-heap.mjs <file>.