mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
fix(challenge): escalate NSFW-text challenges — cancel green, raise R on yellow (#3211)
* docs(challenge): spec for NSFW text-scan escalation + green→yellow flip Design for escalating a user challenge to R and flipping green→yellow when its text scans as sexual content, plus currency-scoping the initial-prize externalTransactionId to prevent a silent-drop unfunded-pool bug on re-charge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(challenge): implementation plan for NSFW scan escalation + flip Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(challenge): currency-scope initial-prize externalTransactionId Refunding a charge leaves its externalTransactionId occupied in the ledger, so a later yellow re-charge on the shared green id would be silently dropped by the createBuzzTransaction dedup, leaving an unfunded pool. Suffix the id with the currency; the -creator prefix matchers still match both variants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(challenge): scan text for suggestive+explicit, not nsfw only The nsfw label (threshold 0.75) misses crude sexual themes; suggestive/explicit (threshold 0.5) catch them with a large margin. Centralize the label set so the adapter submit and scanUserChallenge stay in sync. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(challenge): add computeNsfwEscalation pure decision Given a scanned challenge, decides the raised allowed/display nsfw levels and whether a green user challenge flips to yellow + refunds its green initial prize. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(challenge): applyChallengeNsfwEscalation IO helper Applies a scan verdict: marks Scanned, raises to R, flips green->yellow, refunds the green initial prize (refund-before-update for crash safety), updates the collection browsing level, and notifies the creator. Idempotent on retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(challenge): delegate scan applyResult to escalation helper The adapter now keeps only the Blocked path; clean/NSFW verdicts route through applyChallengeNsfwEscalation, treating any triggered label as NSFW. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(challenge): broaden flip refund prefix + harden collection update - Flip refund uses the -creator prefix so it also matches pre-deploy charge ids (-creator, no currency suffix); the narrow -creator-green prefix missed them and stranded the creator's escrow while zeroing the pool. - collection.updateMany instead of update so a deleted collection no-ops rather than poison-pilling webhook retries with P2025. - Refresh the adapter's stale top-of-file comment for the delegated flip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(challenge): pivot design from green→yellow flip to cancel/void Cancelling a green NSFW challenge (via the existing race-safe voidChallenge) instead of flipping it to yellow reuses battle-tested code and eliminates the currency-migration edge cases. Yellow challenges still raise to R. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(challenge): cancel green NSFW challenges instead of flipping to yellow Green user challenges whose text scans NSFW are now voided (Cancelled + collection closed + initial prize refunded via the race-safe voidChallenge) and the creator is notified to recreate on civitai.red. Yellow challenges still raise to R and stay live. Replaces the buzzType-flip + green-prize-refund path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs+test(challenge): refresh adapter comment for cancel; add already-R escalation case Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(challenge): correct cancel-path ordering to void-before-Scanned Match the spec to the implemented (crash-safe) order: void first so a crash leaves the challenge Cancelled/hidden, never Scanned-and-visible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(challenge): guard blocked-path against deleted challenge + align stale docs - adapter blocked path: early-return when the challenge was deleted between scan submit and the webhook, instead of a bare update that throws P2025 and fails the moderation callback (restores pre-refactor behavior). - spec title + plan header: flag the flip→cancel pivot so the docs don't read as though the code is wrong. Addresses Copilot PR review feedback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(challenge): scan nsfw label only (suggestive/explicit not yet reliable) Only the nsfw XGuard label is currently trustworthy, so scan for it alone. Trade-off: nsfw's 0.75 threshold misses borderline sexual text (~0.68) — only clearly-NSFW text escalates until suggestive/explicit are reliable. Escalation logic is label-agnostic; this is a one-const change. * docs(challenge): document text-scan NSFW handling in feature doc; drop session spec/plan Add a Text moderation bullet to the challenge-platform Safety/gating section (green NSFW → void+refund, yellow → raise to R, nsfw-label-only interim). Remove the superpowers/ spec + plan — implementation scratch that doesn't belong in main. * docs: remove superpowers/specs scratch (article-rating #2779, public-challenges #2965) Implementation-detail specs that don't belong in main; removed per maintainer request alongside this PR's own doc cleanup. * chore: gitignore docs/superpowers/ (skill scratch, not shipped to main) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Manuel Emilio Urena
committed by
GitHub
GitHub
parent
e240c2cc40
commit
0ddd82b64a
@@ -159,3 +159,6 @@ mockups/
|
||||
# local-only SQL (never committed)
|
||||
*.local.sql
|
||||
*.local.md
|
||||
|
||||
# superpowers skill scratch (specs/plans) — implementation detail, not shipped to main
|
||||
docs/superpowers/
|
||||
|
||||
@@ -215,6 +215,7 @@ Idempotency-key prefixes: entry fee `challenge-entry-fee-${challengeId}-${imageI
|
||||
|
||||
### Safety / gating
|
||||
- **Scan gate** — `getChallengeDetail(id, viewerId?)` returns `null` when `source === 'User' && scanStatus !== 'Scanned' && createdById !== viewer`. Activation excludes unscanned user challenges; a `Blocked` user challenge past its start is **auto-voided + refunded**.
|
||||
- **Text moderation** — author text (title/theme/description/invitation) is scanned via XGuard on create + on any text edit (`scanUserChallenge` → `challengeModerationAdapter`). A `blocked` verdict hides the challenge (ingestion `Blocked`). An NSFW verdict escalates via `applyChallengeNsfwEscalation`: a **green** (`buzzType=green`, safe-site) user challenge is **voided** (`voidChallenge` — Cancelled + collection closed + initial prize refunded) and the creator is notified to recreate on civitai.red — green challenges must be SFW; a **yellow** challenge is instead raised to R (add the R bit + collection `forcedBrowsingLevel`) and stays live. Only the `nsfw` label is scanned for now (`suggestive`/`explicit` pending reliability), so nsfw's 0.75 threshold lets borderline text through.
|
||||
- **Browsing level** — user challenges are not clamped to SFW; `InputContentRatingSelect` (defaults SFW, user-selectable) drives `allowedNsfwLevel` (1–63). NSFW isolation (real-cover `browsingLevel` exclusion + client `<Gated>` soft-gate) is handled in the blocker-fix work.
|
||||
- **Orphan guard** — cover `createImage` runs after eligibility assertion, so an ineligible caller leaves no orphan Image.
|
||||
- **Null-safe deleted creator** — `createdById` is nullable (`ON DELETE SET NULL`); `buildChallengeDetail` falls back to system user (`?? -1`).
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
# Article Rating Review — single-action card
|
||||
|
||||
**Date:** 2026-06-25
|
||||
**Branch:** `feat/article-rating-review-single-action`
|
||||
|
||||
## Problem
|
||||
|
||||
The moderator article-rating-review card (`src/pages/moderator/article-rating-review.tsx` →
|
||||
`ArticleRatingReviewCard.tsx`) exposes a level selector plus two buttons (**Approve** / **Reject**).
|
||||
When the mod-selected level equals the system's current level, "Approve as X" and "Reject" produce
|
||||
the same visible rating, differing only in (a) whether an override lock is set and (b) which
|
||||
notification fires. Mods read the two buttons as redundant — confusing.
|
||||
|
||||
Quote: *"if the user suggest PG for their R-rated article... And then I press R, I can either approve
|
||||
as R or reject... Both essentially do the same thing I guess? But probably send different
|
||||
notifications?"*
|
||||
|
||||
## Decision
|
||||
|
||||
Collapse to a **single submit action**. Every resolution pins (overrides) the article at the
|
||||
mod-selected level. Status and notification are **derived** from selected-vs-suggested level.
|
||||
|
||||
- Mod picks one level — segmented control with **no default**. Submit disabled until a pick.
|
||||
- Submit always runs the override/pin write (today's `Actioned` branch): sets
|
||||
`moderatorNsfwLevel = nsfwLevel = appliedLevel`, snapshots `moderatorNsfwLevelBasis`, locks
|
||||
`userNsfwLevel`.
|
||||
- **Server derives status** (authoritative, not client-supplied):
|
||||
- `appliedLevel === suggestedLevel` → `Actioned` (dashboard "Approved") → `approved` notification.
|
||||
- `appliedLevel !== suggestedLevel` → `Unactioned` (dashboard "Rejected") → `rejected` notification.
|
||||
- Button label derives client-side: `Approve as {level}` when pick == owner suggestion, else
|
||||
`Set rating to {level}`.
|
||||
|
||||
Dashboard `Pending / Approved / Rejected` filters + counts are unchanged structurally; their meaning
|
||||
shifts to "was the owner's suggestion granted?" rather than "did we touch the article?".
|
||||
|
||||
## Changes by file
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `src/server/schema/article.schema.ts` | `resolveArticleRatingReviewSchema`: drop `status`, make `appliedLevel` required (positive int), remove the `.refine`. |
|
||||
| `src/server/services/article.service.ts` (`resolveArticleRatingReview`) | Input `{ reviewId, appliedLevel, modComment, moderatorId }`. Read `suggestedLevel` in the txn. **Always** run the override/pin write. Derive `status` from `appliedLevel === suggestedLevel`. Branch notification on derived status; pass `appliedLevel` into the rejected notification details. Return derived `status`. |
|
||||
| `src/server/notifications/article-rating-review.notifications.ts` | Reword the `rejected` `prepareMessage` — a level is now applied, so: `Your rating dispute on "{title}" was reviewed — a moderator set the rating to {appliedLevel}.` Update details type (`appliedLevel` instead of/in addition to `currentLevel`). |
|
||||
| `src/server/routers/article.router.ts` | Pass-through; tracking still receives derived `status` + `appliedLevel`. |
|
||||
| `src/components/Article/ArticleRatingReviewCard.tsx` | Empty-default segmented control; single Submit (disabled until pick + while pending); derived label; one `handleResolve`; update helper text; remove the dual Approve/Reject buttons + `handleApprove`/`handleDismiss`. |
|
||||
| `src/pages/moderator/article-rating-review.tsx` | No logic change. (Optional tooltip clarifying Approved = suggestion granted / Rejected = overrode differently — skip unless trivial.) |
|
||||
|
||||
## Behavior shifts (intended; documented)
|
||||
|
||||
1. **Reject now pins.** Previously `Unactioned` left the rating *floating* (no override). Now it sets
|
||||
an override + basis, so the article won't auto-lower until a mod clears it, and a future
|
||||
down-direction re-dispute routes through `evaluateAutoApproveGate` gate #6 (`derived < basis`)
|
||||
instead of auto-approving freely on rescan. Direct consequence of "always pin".
|
||||
2. **`status` no longer implies "override exists".** Both resolution paths now set the override;
|
||||
`status` only encodes "was the owner's suggestion granted". Verified the only `status='Actioned'`
|
||||
DB predicate near this code is on `Report` (NSFW report), not `ArticleRatingReview` — re-grep
|
||||
during implementation to confirm nothing else keys off review status as an override proxy.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- No change to the auto-approve / re-dispute helper logic itself.
|
||||
- No change to dispute creation (`createArticleRatingReview`).
|
||||
- No backfill of historical `Unactioned` rows (they remain unpinned; only new resolutions pin).
|
||||
@@ -1,200 +0,0 @@
|
||||
# Surface challenge-create requirements on `/challenges/create`
|
||||
|
||||
**Date:** 2026-07-15
|
||||
**Branch base:** `feat/public-challenges`
|
||||
**Status:** Design approved, pending spec review
|
||||
|
||||
## Problem
|
||||
|
||||
Creating a public (User-source) challenge is gated behind a set of eligibility
|
||||
checks — most notably a minimum creator score of 5,000 — but **none of these
|
||||
requirements are surfaced to the user before they submit**. The "Create a
|
||||
Challenge" CTA (challenges index page and user menu) is shown to everyone the
|
||||
feature flags allow, and the user only discovers the requirement reactively:
|
||||
they fill out the entire create form, submit, and the `upsertUserChallenge`
|
||||
mutation throws, surfacing a backend error string as a toast
|
||||
(`ChallengeUpsertForm.tsx:352`). "Find out you're not eligible by failing" is
|
||||
poor UX.
|
||||
|
||||
## Goal
|
||||
|
||||
On `/challenges/create`, proactively show an ineligible user *why* they can't
|
||||
create a challenge — mirroring the existing "Join the Creator Program"
|
||||
requirements card in the buzz dashboard — instead of letting them fill out a
|
||||
form they can't submit.
|
||||
|
||||
## Current state (verified)
|
||||
|
||||
### Enforcement gates (create-only)
|
||||
All live in `src/server/services/challenge-eligibility.service.ts`. Create
|
||||
requires **all** of the following; each `assert*` throws on the **first**
|
||||
failure (no structured "all requirements + status" evaluator exists):
|
||||
|
||||
| Gate | Rule | Source |
|
||||
|---|---|---|
|
||||
| Creator score | `User.meta.scores.total` ≥ `CHALLENGE_MIN_CREATOR_SCORE` (5,000) | `assertUserInGoodStanding:62-68` |
|
||||
| Account standing | not banned/deleted, not muted, 0 active strikes | `assertUserAccountInGoodStanding:49-59` |
|
||||
| Daily create limit | < `CHALLENGE_CREATE_DAILY_LIMIT` User-source challenges in last 24h | `assertUnderDailyCreateLimit:74-90` |
|
||||
| Active challenge limit | < `getChallengeActiveLimit(tier)` Scheduled/Active challenges | `assertUnderActiveChallengeLimit:93-113` |
|
||||
|
||||
`assertCanCreateUserChallenge(userId)` (`:116-120`) chains all three, invoked
|
||||
from `challenge.service.ts:1411` only when creating (no `id`). Constants live in
|
||||
`src/shared/constants/challenge.constants.ts`.
|
||||
|
||||
### Score is not available client-side
|
||||
`CurrentUser` (`CivitaiSessionProvider`) carries no score field, and there is no
|
||||
challenge eligibility tRPC query. So the client currently cannot know the user's
|
||||
score without a new backend call.
|
||||
|
||||
### Reuse reference — Creator Program card
|
||||
The buzz dashboard already solves the analogous problem:
|
||||
- `useCreatorProgramRequirements()` (`CreatorProgram.util.ts:10`) →
|
||||
`trpc.creatorProgram.getCreatorRequirements` returns `{ score: { min, current } }`.
|
||||
- Server handler `getCreatorRequirements` (`creator-program.service.ts:207-242`)
|
||||
reads `User.meta.scores` but computes `current = GREATEST(sum of components, total)`
|
||||
with threshold `MIN_CREATOR_SCORE = 40000`.
|
||||
- `CreatorProgramRequirement({ title, content, isMet })`
|
||||
(`CreatorProgramV2.tsx:307`) renders generic green-check / red-X rows.
|
||||
- `openCreatorScoreModal()` (`CreatorProgramV2.modals.tsx:236`, exported) opens a
|
||||
"What is your Creator Score?" explainer.
|
||||
|
||||
## Design
|
||||
|
||||
### 1. Backend — read-only eligibility evaluator + query
|
||||
|
||||
Add a **non-throwing** evaluator to `challenge-eligibility.service.ts`:
|
||||
|
||||
```ts
|
||||
export type ChallengeCreateRequirement =
|
||||
| { key: 'score'; met: boolean; current: number; min: number }
|
||||
| { key: 'standing'; met: boolean; muted: boolean; activeStrikes: number; banned: boolean }
|
||||
| { key: 'dailyLimit'; met: boolean; recentCount: number; limit: number }
|
||||
| { key: 'activeLimit'; met: boolean; activeCount: number; limit: number };
|
||||
|
||||
export type ChallengeCreateEligibility = {
|
||||
canCreate: boolean;
|
||||
requirements: ChallengeCreateRequirement[];
|
||||
};
|
||||
|
||||
export async function getUserChallengeCreateEligibility(
|
||||
userId: number
|
||||
): Promise<ChallengeCreateEligibility>;
|
||||
```
|
||||
|
||||
It runs the same four checks as the `assert*` path — reusing
|
||||
`getUserChallengeStanding` (same `meta.scores.total` source), the same
|
||||
`dbRead.challenge.count` daily/active queries, `getHighestTierSubscription`, and
|
||||
the **same shared constants** — but returns statuses instead of throwing.
|
||||
`canCreate` is `requirements.every(r => r.met)`.
|
||||
|
||||
**Single source of truth:** the `assert*` functions remain the enforcement
|
||||
boundary and continue to gate the mutation (belt-and-suspenders). Because both
|
||||
paths read the same standing/count helpers and the same constants, the displayed
|
||||
requirements cannot drift from what is enforced. Where practical, factor the
|
||||
per-check predicate (e.g. score-met, under-daily-limit) into a small shared
|
||||
helper consumed by both the evaluator and its matching `assert*`, so the two
|
||||
stay in lockstep without duplicating threshold logic.
|
||||
|
||||
Expose via the router (`challenge.router.ts`):
|
||||
|
||||
```ts
|
||||
getCreateEligibility: protectedProcedure
|
||||
.use(isFlagProtected('challengePlatform'))
|
||||
.use(isFlagProtected('userChallenges'))
|
||||
.query(({ ctx }) => getUserChallengeCreateEligibility(ctx.user.id)),
|
||||
```
|
||||
|
||||
No zod input (uses `ctx.user.id`), matching the other user-challenge routes'
|
||||
flag guards.
|
||||
|
||||
### 2. Score source — accepted divergence
|
||||
|
||||
The challenge card shows `meta.scores.total` **alone** (threshold 5,000) because
|
||||
that is exactly what the create gate enforces — display must equal enforcement.
|
||||
The Creator Program card shows `GREATEST(components_sum, total)` (threshold
|
||||
40,000). The two cards can therefore show slightly different score numbers for
|
||||
the same user. Truthfulness to the actual gate wins; this is an accepted
|
||||
tradeoff. Aligning the challenge gate to `GREATEST(...)` is a policy change and
|
||||
is out of scope for this work.
|
||||
|
||||
### 3. Frontend — block on ineligibility
|
||||
|
||||
New component `src/components/Challenge/ChallengeCreateRequirements.tsx`:
|
||||
- Presentational card given a `ChallengeCreateEligibility`.
|
||||
- Header, e.g. "Requirements to create a challenge".
|
||||
- Renders a **challenge-specific** requirement row component (a small local
|
||||
component in the same file — *not* a reuse of `CreatorProgramRequirement`, per
|
||||
product decision) with the green-check / red-X treatment.
|
||||
- Rows: score, standing, and active-limit render always (each met/unmet). The
|
||||
**daily-create-limit** row is an anti-spam throttle, not a standing entitlement
|
||||
— showing "5/day allowed" beside the "1 active" cap reads as a contradiction,
|
||||
so it renders only when it is the actual blocker (unmet). The evaluator still
|
||||
computes it and `canCreate` still accounts for it.
|
||||
- **Creator Score** — headline row. Title: `Have a creator score of at least 5,000`.
|
||||
Content: `Your current Creator Score is {abbreviateNumber(current)}.` with
|
||||
"Creator Score" rendered as a link that calls `openCreatorScoreModal()`
|
||||
(imported from `CreatorProgramV2.modals`) — the explainer link.
|
||||
- **Account in good standing** — content reflects the failing sub-reason
|
||||
(muted / active strikes / banned) or "good standing" when met.
|
||||
- **Daily create limit** — content: `You've created {recentCount} of {limit}
|
||||
challenges allowed in the last 24 hours.`
|
||||
- **Active challenge limit** — content: `You have {activeCount} of {limit}
|
||||
active challenges for your membership tier.`
|
||||
|
||||
Wire into `src/pages/challenges/create.tsx`:
|
||||
|
||||
```tsx
|
||||
const currentUser = useCurrentUser();
|
||||
const { data: eligibility, isLoading } =
|
||||
trpc.challenge.getCreateEligibility.useQuery(undefined, { enabled: !!currentUser });
|
||||
|
||||
if (isLoading) return <PageLoader />;
|
||||
if (eligibility && !eligibility.canCreate)
|
||||
return <ChallengeCreateRequirements eligibility={eligibility} />;
|
||||
return <ChallengeUpsertForm variant="user" />;
|
||||
```
|
||||
|
||||
Existing SSR flag guards (`create.tsx:31`) are unchanged.
|
||||
|
||||
**Query-error fallback:** if `getCreateEligibility` errors (not merely
|
||||
ineligible), render the form. The backend `assert*` gate still enforces on
|
||||
submit, degrading gracefully to today's error-toast behavior rather than hard-
|
||||
blocking a possibly-eligible user on a transient read failure.
|
||||
|
||||
### 4. Scope
|
||||
|
||||
- Index-page "Create Challenge" CTA and the user-menu "Create a Challenge" item
|
||||
stay **ungated** on eligibility. Clicking either routes to `/challenges/create`,
|
||||
which now shows the blocking requirements card for ineligible users. No change
|
||||
to those entry points.
|
||||
- No change to the enforcement path or the mutation's error handling; the card
|
||||
is additive.
|
||||
|
||||
## Testing
|
||||
|
||||
Extend `src/server/services/__tests__/challenge-eligibility.service.test.ts`:
|
||||
- `getUserChallengeCreateEligibility` returns `canCreate: true` with every row
|
||||
`met` for an eligible user.
|
||||
- Score below threshold → `canCreate: false`, only the `score` row unmet, with
|
||||
correct `current`/`min`.
|
||||
- Muted / active strikes / banned → `standing` row unmet.
|
||||
- Daily limit reached → `dailyLimit` row unmet with `recentCount`/`limit`.
|
||||
- Active limit reached → `activeLimit` row unmet with `activeCount`/`limit`.
|
||||
- **Parity test:** for a representative matrix of standings, the evaluator's
|
||||
`canCreate` equals "`assertCanCreateUserChallenge` does not throw", guarding
|
||||
against display/enforcement drift.
|
||||
|
||||
## Files touched
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `src/server/services/challenge-eligibility.service.ts` | add `getUserChallengeCreateEligibility` + types; optionally factor shared per-check predicates |
|
||||
| `src/server/routers/challenge.router.ts` | add `getCreateEligibility` query |
|
||||
| `src/components/Challenge/ChallengeCreateRequirements.tsx` | new card + challenge-specific requirement row |
|
||||
| `src/pages/challenges/create.tsx` | fetch eligibility; block with card vs. render form |
|
||||
| `src/server/services/__tests__/challenge-eligibility.service.test.ts` | evaluator + parity tests |
|
||||
|
||||
## Out of scope
|
||||
- Aligning the challenge score source to Creator Program's `GREATEST(...)`.
|
||||
- Gating the index/user-menu CTAs on eligibility.
|
||||
- Any change to the enforcement gates themselves.
|
||||
@@ -1,10 +1,12 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { NsfwLevel } from '~/server/common/enums';
|
||||
import { nsfwBrowsingLevelsFlag, sfwBrowsingLevelsFlag } from '~/shared/constants/browsingLevel.constants';
|
||||
import { ChallengeSource } from '~/shared/utils/prisma/enums';
|
||||
import {
|
||||
deriveDomainCurrency,
|
||||
isNonSfwForGreen,
|
||||
isChallengeHiddenByDomainCurrency,
|
||||
computeNsfwEscalation,
|
||||
} from './challenge-currency';
|
||||
|
||||
describe('deriveDomainCurrency', () => {
|
||||
@@ -60,3 +62,68 @@ describe('isChallengeHiddenByDomainCurrency', () => {
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('computeNsfwEscalation', () => {
|
||||
const PG_PG13 = NsfwLevel.PG | NsfwLevel.PG13; // 3, SFW mask
|
||||
|
||||
it('clean scan: nsfwLevel = derived base, no cancel', () => {
|
||||
const r = computeNsfwEscalation({
|
||||
allowedNsfwLevel: PG_PG13,
|
||||
buzzType: 'green',
|
||||
source: ChallengeSource.User,
|
||||
isNsfw: false,
|
||||
});
|
||||
expect(r.allowedNsfwLevel).toBe(PG_PG13);
|
||||
expect(r.nsfwLevel).toBe(NsfwLevel.PG13);
|
||||
expect(r.cancel).toBe(false);
|
||||
});
|
||||
|
||||
it('green user challenge + nsfw: cancel, level left unchanged', () => {
|
||||
const r = computeNsfwEscalation({
|
||||
allowedNsfwLevel: PG_PG13,
|
||||
buzzType: 'green',
|
||||
source: ChallengeSource.User,
|
||||
isNsfw: true,
|
||||
});
|
||||
expect(r.cancel).toBe(true);
|
||||
expect(r.allowedNsfwLevel).toBe(PG_PG13);
|
||||
expect(r.nsfwLevel).toBe(NsfwLevel.PG13);
|
||||
});
|
||||
|
||||
it('yellow user challenge + nsfw: raise to R, no cancel', () => {
|
||||
const r = computeNsfwEscalation({
|
||||
allowedNsfwLevel: PG_PG13,
|
||||
buzzType: 'yellow',
|
||||
source: ChallengeSource.User,
|
||||
isNsfw: true,
|
||||
});
|
||||
expect(r.cancel).toBe(false);
|
||||
expect(r.allowedNsfwLevel).toBe(PG_PG13 | NsfwLevel.R);
|
||||
expect(r.nsfwLevel).toBe(NsfwLevel.R);
|
||||
});
|
||||
|
||||
it('non-user (System) green challenge + nsfw: raise to R, never cancel', () => {
|
||||
const r = computeNsfwEscalation({
|
||||
allowedNsfwLevel: PG_PG13,
|
||||
buzzType: 'green',
|
||||
source: ChallengeSource.System,
|
||||
isNsfw: true,
|
||||
});
|
||||
expect(r.cancel).toBe(false);
|
||||
expect(r.allowedNsfwLevel).toBe(PG_PG13 | NsfwLevel.R);
|
||||
expect(r.nsfwLevel).toBe(NsfwLevel.R);
|
||||
});
|
||||
|
||||
it('yellow challenge already at R + nsfw: idempotent, stays at R (no cancel)', () => {
|
||||
const alreadyR = NsfwLevel.PG | NsfwLevel.PG13 | NsfwLevel.R; // 7
|
||||
const r = computeNsfwEscalation({
|
||||
allowedNsfwLevel: alreadyR,
|
||||
buzzType: 'yellow',
|
||||
source: ChallengeSource.User,
|
||||
isNsfw: true,
|
||||
});
|
||||
expect(r.cancel).toBe(false);
|
||||
expect(r.allowedNsfwLevel).toBe(alreadyR);
|
||||
expect(r.nsfwLevel).toBe(NsfwLevel.R);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
import { nsfwBrowsingLevelsFlag } from '~/shared/constants/browsingLevel.constants';
|
||||
import { NsfwLevel } from '~/server/common/enums';
|
||||
import { Flags } from '~/shared/utils/flags';
|
||||
import { ChallengeSource } from '~/shared/utils/prisma/enums';
|
||||
import { deriveChallengeNsfwLevel } from '~/server/games/daily-challenge/daily-challenge.utils';
|
||||
|
||||
export type ChallengeBuzzType = 'green' | 'yellow';
|
||||
|
||||
@@ -35,3 +37,36 @@ export function isChallengeHiddenByDomainCurrency(
|
||||
if (challenge.createdById != null && challenge.createdById === viewerId) return false;
|
||||
return challenge.buzzType !== deriveDomainCurrency(isGreen);
|
||||
}
|
||||
|
||||
export type NsfwEscalation = {
|
||||
allowedNsfwLevel: number;
|
||||
nsfwLevel: number;
|
||||
cancel: boolean;
|
||||
};
|
||||
|
||||
// Decide how a scanned challenge escalates. A clean scan just recomputes the display level from the
|
||||
// (unchanged) allowed mask. An NSFW verdict on a green USER challenge cancels it (green must be SFW —
|
||||
// the caller voids + refunds). An NSFW verdict on a yellow/non-user challenge raises the rating to R so
|
||||
// it drops out of safe feeds while staying live.
|
||||
export function computeNsfwEscalation(input: {
|
||||
allowedNsfwLevel: number;
|
||||
buzzType: ChallengeBuzzType;
|
||||
source: ChallengeSource;
|
||||
isNsfw: boolean;
|
||||
}): NsfwEscalation {
|
||||
const cancel =
|
||||
input.isNsfw && input.source === ChallengeSource.User && input.buzzType === 'green';
|
||||
if (!input.isNsfw || cancel) {
|
||||
return {
|
||||
allowedNsfwLevel: input.allowedNsfwLevel,
|
||||
nsfwLevel: deriveChallengeNsfwLevel(input.allowedNsfwLevel),
|
||||
cancel,
|
||||
};
|
||||
}
|
||||
const allowedNsfwLevel = Flags.addFlag(input.allowedNsfwLevel, NsfwLevel.R);
|
||||
return {
|
||||
allowedNsfwLevel,
|
||||
nsfwLevel: deriveChallengeNsfwLevel(allowedNsfwLevel),
|
||||
cancel: false,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -251,6 +251,25 @@ describe('chargeInitialPrize fromAccountType', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('chargeInitialPrize externalTransactionId', () => {
|
||||
beforeEach(() => {
|
||||
mockCreateBuzzTransaction.mockReset();
|
||||
mockCreateBuzzTransaction.mockResolvedValue({ transactionId: 'tx-1' });
|
||||
});
|
||||
|
||||
it('scopes the externalTransactionId by currency (green)', async () => {
|
||||
await chargeInitialPrize({ challengeId: 42, userId: 7, amount: 100, fromAccountType: 'green' });
|
||||
const [arg] = mockCreateBuzzTransaction.mock.calls[0];
|
||||
expect(arg.externalTransactionId).toBe('challenge-initial-prize-42-creator-green');
|
||||
});
|
||||
|
||||
it('scopes the externalTransactionId by currency (yellow)', async () => {
|
||||
await chargeInitialPrize({ challengeId: 42, userId: 7, amount: 100, fromAccountType: 'yellow' });
|
||||
const [arg] = mockCreateBuzzTransaction.mock.calls[0];
|
||||
expect(arg.externalTransactionId).toBe('challenge-initial-prize-42-creator-yellow');
|
||||
});
|
||||
});
|
||||
|
||||
describe('chargeEntryFees fromAccountType', () => {
|
||||
it('forwards fromAccountType to both house and pool legs', async () => {
|
||||
mockCreateBuzzTransactionMany
|
||||
|
||||
@@ -72,9 +72,12 @@ export async function chargeInitialPrize({
|
||||
type: TransactionType.Purchase,
|
||||
amount,
|
||||
description: 'Challenge initial prize pool',
|
||||
// Trailing non-numeric token keeps the completion refund's startsWith prefix match unambiguous
|
||||
// vs other challenge ids (challenge 5 would otherwise prefix-match 50, 51, ...).
|
||||
externalTransactionId: `challenge-initial-prize-${challengeId}-creator`,
|
||||
// Trailing `-creator` keeps prefix matches unambiguous vs other challenge ids (challenge 5 would
|
||||
// otherwise prefix-match 50, 51, ...). The currency suffix scopes the id per wallet: a refunded
|
||||
// green charge leaves its id occupied in the ledger, so a later yellow re-charge on a shared id
|
||||
// would be silently dropped (createBuzzTransaction dedups on externalTransactionId) — leaving an
|
||||
// unfunded pool. `-creator` prefix matchers still match both `-creator-green` and `-creator-yellow`.
|
||||
externalTransactionId: `challenge-initial-prize-${challengeId}-creator-${fromAccountType}`,
|
||||
details: { challengeId },
|
||||
});
|
||||
log(`Escrowed ${amount} buzz initial prize for challenge ${challengeId}`);
|
||||
|
||||
@@ -33,6 +33,12 @@ import {
|
||||
// DB/Redis into client bundles)
|
||||
export { computeDynamicPool, distributePrizes } from './challenge-pool';
|
||||
|
||||
// Labels requested for the challenge text scan. Only `nsfw` is currently reliable in XGuard, so we
|
||||
// scan for it alone; `suggestive`/`explicit` are omitted until they're trustworthy. Trade-off: nsfw's
|
||||
// 0.75 threshold misses borderline sexual text (a theme scoring ~0.68 slips through) — only clearly
|
||||
// NSFW text escalates for now. Any triggered label counts as NSFW downstream.
|
||||
export const CHALLENGE_MODERATION_LABELS = ['nsfw'] as const;
|
||||
|
||||
// Author-supplied text sent to the text-moderation scan. Description is RTE HTML, so tags are
|
||||
// stripped.
|
||||
export function buildChallengeModerationText(challenge: {
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
|
||||
const { mockDbRead, mockDbWrite, mockVoidChallenge, mockCreateNotification } = vi.hoisted(() => ({
|
||||
mockDbRead: {
|
||||
challenge: { findUnique: vi.fn() },
|
||||
collection: { findUnique: vi.fn() },
|
||||
},
|
||||
mockDbWrite: {
|
||||
challenge: { update: vi.fn() },
|
||||
collection: { updateMany: vi.fn() },
|
||||
},
|
||||
mockVoidChallenge: vi.fn(),
|
||||
mockCreateNotification: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('~/server/db/client', () => ({ dbRead: mockDbRead, dbWrite: mockDbWrite }));
|
||||
vi.mock('~/server/services/challenge.service', () => ({ voidChallenge: mockVoidChallenge }));
|
||||
vi.mock('~/server/services/notification.service', () => ({
|
||||
createNotification: mockCreateNotification,
|
||||
}));
|
||||
|
||||
const { applyChallengeNsfwEscalation } = await import('./challenge-nsfw-escalation');
|
||||
|
||||
const PG_PG13 = 3; // NsfwLevel.PG | NsfwLevel.PG13
|
||||
const R = 4;
|
||||
|
||||
function challenge(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
allowedNsfwLevel: PG_PG13,
|
||||
buzzType: 'green',
|
||||
source: 'User',
|
||||
createdById: 7,
|
||||
collectionId: 55,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
mockDbWrite.challenge.update.mockResolvedValue({});
|
||||
mockDbWrite.collection.updateMany.mockResolvedValue({ count: 1 });
|
||||
mockDbRead.collection.findUnique.mockResolvedValue({ metadata: { forcedBrowsingLevel: PG_PG13 } });
|
||||
mockVoidChallenge.mockResolvedValue({ success: true });
|
||||
mockCreateNotification.mockResolvedValue(undefined);
|
||||
});
|
||||
|
||||
describe('applyChallengeNsfwEscalation', () => {
|
||||
it('clean scan: marks Scanned, no void, no level raise, no collection update', async () => {
|
||||
mockDbRead.challenge.findUnique.mockResolvedValue(challenge());
|
||||
await applyChallengeNsfwEscalation({ entityId: 1, isNsfw: false });
|
||||
|
||||
const data = mockDbWrite.challenge.update.mock.calls[0][0].data;
|
||||
expect(data.ingestion).toBe('Scanned');
|
||||
expect(data.nsfwLevel).toBe(2); // PG13
|
||||
expect(data.allowedNsfwLevel).toBe(PG_PG13);
|
||||
expect(mockVoidChallenge).not.toHaveBeenCalled();
|
||||
expect(mockDbWrite.collection.updateMany).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('green user + nsfw: voids BEFORE marking Scanned, notifies cancelled, no level raise/collection update', async () => {
|
||||
mockDbRead.challenge.findUnique.mockResolvedValue(challenge());
|
||||
const order: string[] = [];
|
||||
mockVoidChallenge.mockImplementation(async () => {
|
||||
order.push('void');
|
||||
return { success: true };
|
||||
});
|
||||
mockDbWrite.challenge.update.mockImplementation(async () => {
|
||||
order.push('update');
|
||||
return {};
|
||||
});
|
||||
|
||||
await applyChallengeNsfwEscalation({ entityId: 42, isNsfw: true });
|
||||
|
||||
expect(order).toEqual(['void', 'update']);
|
||||
expect(mockVoidChallenge).toHaveBeenCalledWith(42);
|
||||
const data = mockDbWrite.challenge.update.mock.calls[0][0].data;
|
||||
expect(data.ingestion).toBe('Scanned');
|
||||
expect(data.nsfwLevel).toBeUndefined(); // cancel path does not raise the level
|
||||
expect(data.buzzType).toBeUndefined();
|
||||
expect(mockDbWrite.collection.updateMany).not.toHaveBeenCalled();
|
||||
expect(mockCreateNotification).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ key: 'challenge-nsfw-cancelled-42' })
|
||||
);
|
||||
});
|
||||
|
||||
it('yellow user + nsfw: raises to R, updates collection, notifies raised, does NOT void', async () => {
|
||||
mockDbRead.challenge.findUnique.mockResolvedValue(challenge({ buzzType: 'yellow' }));
|
||||
await applyChallengeNsfwEscalation({ entityId: 9, isNsfw: true });
|
||||
|
||||
expect(mockVoidChallenge).not.toHaveBeenCalled();
|
||||
const data = mockDbWrite.challenge.update.mock.calls[0][0].data;
|
||||
expect(data.allowedNsfwLevel).toBe(PG_PG13 | R);
|
||||
expect(data.nsfwLevel).toBe(R);
|
||||
const colData = mockDbWrite.collection.updateMany.mock.calls[0][0].data;
|
||||
expect(colData.metadata.forcedBrowsingLevel).toBe(PG_PG13 | R);
|
||||
expect(mockCreateNotification).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ key: 'challenge-nsfw-raised-9' })
|
||||
);
|
||||
});
|
||||
|
||||
it('missing challenge: no-op', async () => {
|
||||
mockDbRead.challenge.findUnique.mockResolvedValue(null);
|
||||
await applyChallengeNsfwEscalation({ entityId: 404, isNsfw: true });
|
||||
expect(mockDbWrite.challenge.update).not.toHaveBeenCalled();
|
||||
expect(mockVoidChallenge).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,127 @@
|
||||
import { NotificationCategory } from '~/server/common/enums';
|
||||
import { dbRead, dbWrite } from '~/server/db/client';
|
||||
import {
|
||||
computeNsfwEscalation,
|
||||
type ChallengeBuzzType,
|
||||
} from '~/server/games/daily-challenge/challenge-currency';
|
||||
import { createNotification } from '~/server/services/notification.service';
|
||||
import { voidChallenge } from '~/server/services/challenge.service';
|
||||
import type { CollectionMetadataSchema } from '~/server/schema/collection.schema';
|
||||
import { ChallengeIngestionStatus, ChallengeSource } from '~/shared/utils/prisma/enums';
|
||||
|
||||
// Applies the scan verdict to a challenge. A green USER challenge whose text is NSFW is cancelled
|
||||
// (green must be SFW): void it (Cancelled + collection closed + initial prize refunded, all idempotent),
|
||||
// mark the scan resolved, and notify the creator to recreate on civitai.red. A yellow/non-user challenge
|
||||
// stays live but has its rating raised to R. A clean scan just marks it Scanned.
|
||||
//
|
||||
// Idempotent: the cancel path relies on voidChallenge's own idempotency (a retried callback re-runs the
|
||||
// no-op refund on the already-Cancelled row) plus a deterministic notification key. buzzType is never
|
||||
// changed, so the cancel decision is stable across retries.
|
||||
export async function applyChallengeNsfwEscalation({
|
||||
entityId,
|
||||
isNsfw,
|
||||
}: {
|
||||
entityId: number;
|
||||
isNsfw: boolean;
|
||||
}): Promise<void> {
|
||||
const challenge = await dbRead.challenge.findUnique({
|
||||
where: { id: entityId },
|
||||
select: {
|
||||
allowedNsfwLevel: true,
|
||||
buzzType: true,
|
||||
source: true,
|
||||
createdById: true,
|
||||
collectionId: true,
|
||||
},
|
||||
});
|
||||
if (!challenge) return;
|
||||
|
||||
const buzzType: ChallengeBuzzType = challenge.buzzType === 'green' ? 'green' : 'yellow';
|
||||
const escalation = computeNsfwEscalation({
|
||||
allowedNsfwLevel: challenge.allowedNsfwLevel,
|
||||
buzzType,
|
||||
source: challenge.source,
|
||||
isNsfw,
|
||||
});
|
||||
|
||||
if (escalation.cancel) {
|
||||
// Void FIRST (Cancelled + collection closed + prize refunded, idempotent) so a crash before the
|
||||
// scan-state write leaves the challenge Cancelled/hidden — never a Scanned-and-therefore-visible
|
||||
// green NSFW challenge. Then resolve the moderation state.
|
||||
await voidChallenge(entityId);
|
||||
await dbWrite.challenge.update({
|
||||
where: { id: entityId },
|
||||
data: { ingestion: ChallengeIngestionStatus.Scanned, scannedAt: new Date() },
|
||||
});
|
||||
if (challenge.createdById) {
|
||||
await createNotification({
|
||||
userId: challenge.createdById,
|
||||
category: NotificationCategory.System,
|
||||
type: 'system-message',
|
||||
key: `challenge-nsfw-cancelled-${entityId}`,
|
||||
details: {
|
||||
message:
|
||||
'Your challenge was cancelled because its text was flagged as adult content — green challenges must be safe-for-work. Any prize you funded has been refunded; you can recreate it on civitai.red.',
|
||||
url: `/challenges/${entityId}`,
|
||||
},
|
||||
});
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
await dbWrite.challenge.update({
|
||||
where: { id: entityId },
|
||||
data: {
|
||||
ingestion: ChallengeIngestionStatus.Scanned,
|
||||
scannedAt: new Date(),
|
||||
nsfwLevel: escalation.nsfwLevel,
|
||||
allowedNsfwLevel: escalation.allowedNsfwLevel,
|
||||
},
|
||||
});
|
||||
|
||||
// Keep the collection's entry-gating level in step with the raised allowed level. updateMany (not
|
||||
// update) so a deleted collection no-ops instead of throwing P2025 on webhook retries.
|
||||
if (isNsfw && challenge.collectionId) {
|
||||
const collection = await dbRead.collection.findUnique({
|
||||
where: { id: challenge.collectionId },
|
||||
select: { metadata: true },
|
||||
});
|
||||
await dbWrite.collection.updateMany({
|
||||
where: { id: challenge.collectionId },
|
||||
data: {
|
||||
metadata: {
|
||||
...(collection?.metadata as CollectionMetadataSchema),
|
||||
forcedBrowsingLevel: escalation.allowedNsfwLevel,
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (
|
||||
isNsfw &&
|
||||
challenge.createdById &&
|
||||
escalation.nsfwLevel > deriveBase(challenge.allowedNsfwLevel)
|
||||
) {
|
||||
await createNotification({
|
||||
userId: challenge.createdById,
|
||||
category: NotificationCategory.System,
|
||||
type: 'system-message',
|
||||
key: `challenge-nsfw-raised-${entityId}`,
|
||||
details: {
|
||||
message:
|
||||
"Your challenge's rating was raised to R based on its text, so it won't appear in safe-mode feeds.",
|
||||
url: `/challenges/${entityId}`,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// The display level implied by the ORIGINAL allowed mask, to detect an actual raise before notifying.
|
||||
function deriveBase(allowedNsfwLevel: number): number {
|
||||
return computeNsfwEscalation({
|
||||
allowedNsfwLevel,
|
||||
buzzType: 'yellow',
|
||||
source: ChallengeSource.User,
|
||||
isNsfw: false,
|
||||
}).nsfwLevel;
|
||||
}
|
||||
@@ -1,11 +1,14 @@
|
||||
import { NotificationCategory, NsfwLevel } from '~/server/common/enums';
|
||||
import { NotificationCategory } from '~/server/common/enums';
|
||||
import { dbRead, dbWrite } from '~/server/db/client';
|
||||
import type { ModerationAdapter } from '~/server/services/entity-moderation.service';
|
||||
import { createNotification } from '~/server/services/notification.service';
|
||||
import { submitTextModeration } from '~/server/services/text-moderation.service';
|
||||
import { buildChallengeModerationText } from '~/server/games/daily-challenge/challenge-helpers';
|
||||
import {
|
||||
buildChallengeModerationText,
|
||||
CHALLENGE_MODERATION_LABELS,
|
||||
} from '~/server/games/daily-challenge/challenge-helpers';
|
||||
import { parseChallengeMetadata } from '~/server/schema/challenge.schema';
|
||||
import { deriveChallengeNsfwLevel } from '~/server/games/daily-challenge/daily-challenge.utils';
|
||||
import { applyChallengeNsfwEscalation } from '~/server/games/daily-challenge/challenge-nsfw-escalation';
|
||||
import { ChallengeIngestionStatus } from '~/shared/utils/prisma/enums';
|
||||
|
||||
// Challenge-side hooks for the EntityModeration pipeline, mirroring the Article adapter. The
|
||||
@@ -14,7 +17,9 @@ import { ChallengeIngestionStatus } from '~/shared/utils/prisma/enums';
|
||||
//
|
||||
// Result resolution (same shape as articles):
|
||||
// - `blocked` → ToS violation: hide the challenge (ingestion Blocked) + notify the creator.
|
||||
// - `nsfw` (not blocked) → keep visible but floor nsfwLevel to R so it drops out of safe feeds.
|
||||
// - not blocked → routed to `applyChallengeNsfwEscalation`, which on an NSFW verdict cancels a
|
||||
// green USER challenge (void + refund + notify to recreate on civitai.red) and raises a
|
||||
// yellow/non-user challenge to R in place.
|
||||
// - clean → visible at the creator's declared level.
|
||||
// Unlike articles, a challenge's nsfwLevel isn't image-derived, so the R floor is written directly
|
||||
// rather than recomputed from a SQL aggregate.
|
||||
@@ -40,23 +45,24 @@ export const challengeModerationAdapter: ModerationAdapter = {
|
||||
entityType: 'Challenge',
|
||||
entityId,
|
||||
content,
|
||||
labels: ['nsfw'],
|
||||
labels: [...CHALLENGE_MODERATION_LABELS],
|
||||
priority: 'low',
|
||||
}),
|
||||
|
||||
applyResult: async ({ entityId, blocked, triggeredLabels }) => {
|
||||
const challenge = await dbRead.challenge.findUnique({
|
||||
where: { id: entityId },
|
||||
select: { allowedNsfwLevel: true, createdById: true },
|
||||
});
|
||||
if (!challenge) return;
|
||||
|
||||
if (blocked) {
|
||||
const challenge = await dbRead.challenge.findUnique({
|
||||
where: { id: entityId },
|
||||
select: { createdById: true },
|
||||
});
|
||||
// Deleted between submit and this webhook — nothing to hide or notify (a bare update would
|
||||
// throw P2025 and fail the moderation callback).
|
||||
if (!challenge) return;
|
||||
await dbWrite.challenge.update({
|
||||
where: { id: entityId },
|
||||
data: { ingestion: ChallengeIngestionStatus.Blocked, scannedAt: new Date() },
|
||||
});
|
||||
if (challenge.createdById) {
|
||||
if (challenge?.createdById) {
|
||||
await createNotification({
|
||||
userId: challenge.createdById,
|
||||
category: NotificationCategory.System,
|
||||
@@ -71,28 +77,8 @@ export const challengeModerationAdapter: ModerationAdapter = {
|
||||
return;
|
||||
}
|
||||
|
||||
const base = deriveChallengeNsfwLevel(challenge.allowedNsfwLevel);
|
||||
const isNsfw = triggeredLabels.some((label) => label.toLowerCase() === 'nsfw');
|
||||
const nsfwLevel = isNsfw ? Math.max(base, NsfwLevel.R) : base;
|
||||
|
||||
await dbWrite.challenge.update({
|
||||
where: { id: entityId },
|
||||
data: { ingestion: ChallengeIngestionStatus.Scanned, scannedAt: new Date(), nsfwLevel },
|
||||
});
|
||||
|
||||
if (isNsfw && nsfwLevel > base && challenge.createdById) {
|
||||
await createNotification({
|
||||
userId: challenge.createdById,
|
||||
category: NotificationCategory.System,
|
||||
type: 'system-message',
|
||||
key: `challenge-nsfw-raised-${entityId}`,
|
||||
details: {
|
||||
message:
|
||||
"Your challenge's rating was raised to R based on its text, so it won't appear in safe-mode feeds.",
|
||||
url: `/challenges/${entityId}`,
|
||||
},
|
||||
});
|
||||
}
|
||||
// Any of nsfw / suggestive / explicit crossing threshold escalates the challenge.
|
||||
await applyChallengeNsfwEscalation({ entityId, isNsfw: triggeredLabels.length > 0 });
|
||||
},
|
||||
|
||||
// Terminal scan failure: mark retryable Error (the scan gate keeps the challenge hidden). The
|
||||
|
||||
@@ -4,6 +4,7 @@ import { dbRead, dbWrite } from '~/server/db/client';
|
||||
import { FLIPT_FEATURE_FLAGS, isFlipt } from '~/server/flipt/client';
|
||||
import { logToAxiom } from '~/server/logging/client';
|
||||
import {
|
||||
CHALLENGE_MODERATION_LABELS,
|
||||
claimChallengeForCompletion,
|
||||
buildChallengeModerationText,
|
||||
closeChallengeCollection,
|
||||
@@ -1751,7 +1752,7 @@ export async function scanUserChallenge(challengeId: number): Promise<void> {
|
||||
...challenge,
|
||||
themeElements: parseChallengeMetadata(challenge.metadata).themeElements,
|
||||
}),
|
||||
labels: ['nsfw'],
|
||||
labels: [...CHALLENGE_MODERATION_LABELS],
|
||||
priority: 'low',
|
||||
});
|
||||
} catch (e) {
|
||||
|
||||
Reference in New Issue
Block a user