mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
feat(app-blocks): make the cap-limit degrade path observable + rename the absolute-ceiling env knobs (#3528)
Two follow-ups to #3519 (per-app generation spend/velocity caps). 1) OBSERVABILITY of the degrade-to-strictest path. `resolveAppCapLimits` falls back to STRICTEST_APP_CAP_LIMITS on a DB error or a missing `app_blocks` row. That behaviour is right — never uncapped, and a hard deny would turn a transient DB blip into a full generation outage — but it was SILENT, and an app pinned to the strictest ceiling looks exactly like an app that is merely busy. The first symptom would be that app's users hitting abuse rejections they did not earn. (This is the same silent- degradation shape as #3520, which is why it is worth closing here.) Adds `civitai_app_block_cap_limits_degraded_total{reason}` — `db_error` (the read threw: infra; every app degrades at once, page-worthy) vs `missing_row` (the read succeeded and there is no such app: one app, points at an id-minting bug) — plus a paired `console.warn` carrying the specific appBlockId. Follows the existing convention in src/server/metrics/app-block-runtime.metrics.ts (get-or-create against the default registry + a fail-soft emit wrapper); no new mechanism. - NO `app_block_id` prom label, deliberately. `missing_row` fires precisely for ids absent from the app catalog, i.e. the unbounded population known-app-blocks.service.ts exists to clamp, and prom-client retains every distinct label set in the heap forever. The usual clamp needs a DB read — the very thing broken on the `db_error` path. So: alert on the metric, attribute from the log. - NOT a failure path. The metric emit and the log are independently guarded and `recordAppCapLimitsDegrade` is total on its own side, so neither a broken registry nor a throwing console can perturb cap resolution. - NOT on the hot path. Only a cache MISS that DEGRADED emits; a warm hit and a miss that resolves a real row never reach it. Volume is bounded by the 5s fallback-TTL cache, not by submit rate — a 10k-submit burst against one degraded app emits once. 2) RENAME the two absolute-ceiling env knobs. `BLOCK_APP_SPEND_CAP_BUZZ_PER_DAY` / `BLOCK_APP_SPEND_VELOCITY_MAX_GENS` used to BE the ceilings. Since #3519 they are absolute bounds that clamp the tier table AND any per-app moderator override, so an operator reaching for `..._VELOCITY_MAX_GENS` mid-incident would reasonably read it as "set the limit" rather than "bound it". BLOCK_APP_SPEND_CAP_BUZZ_PER_DAY -> BLOCK_APP_SPEND_ABSOLUTE_MAX_BUZZ_PER_DAY BLOCK_APP_SPEND_VELOCITY_MAX_GENS -> BLOCK_APP_SPEND_ABSOLUTE_MAX_GENS_PER_WINDOW The legacy names are still honoured (dp-prod sets neither, but other environments are not enumerable from here, and silently ignoring a set spend guardrail is unacceptable), with a deprecation warning that also states the changed meaning. A valid new value always wins; a set-but-ignored or set-but-unusable legacy value warns too. Exported symbols follow the env names, with the old export names kept as deprecated aliases so pre-rename importers keep compiling. Tests: 41 new across two files. Every new guard was mutation-verified — 14 mutations, each killing a specific named test.
This commit is contained in:
@@ -198,3 +198,15 @@ FLIPT_URL=""
|
||||
FLIPT_FETCHER_SECRET=placeholder
|
||||
|
||||
IMAGE_SCANNER_NEW=false
|
||||
|
||||
# App Blocks — per-app generation spend/velocity ABSOLUTE CEILINGS (incident knobs).
|
||||
# 🔴 These are UPPER BOUNDS, not the limit an app receives. Each app's actual
|
||||
# ceilings come from its server-owned `spendTier` (+ any moderator per-app
|
||||
# override); these clamp the tier table AND any override from above, so setting
|
||||
# one TIGHTENS every app and can never loosen one. Unset = no extra clamp.
|
||||
# Formerly BLOCK_APP_SPEND_CAP_BUZZ_PER_DAY / BLOCK_APP_SPEND_VELOCITY_MAX_GENS —
|
||||
# those names are DEPRECATED but still honoured (with a startup warning).
|
||||
# BLOCK_APP_SPEND_ABSOLUTE_MAX_BUZZ_PER_DAY=
|
||||
# BLOCK_APP_SPEND_ABSOLUTE_MAX_GENS_PER_WINDOW=
|
||||
# Window (seconds) the gens-per-window ceiling is measured over. Default 60.
|
||||
# BLOCK_APP_SPEND_VELOCITY_WINDOW_SECONDS=
|
||||
|
||||
Reference in New Issue
Block a user