ClickUp 868m5wp5r. Image and video ingestion can submit the single
orchestrator `imageScanning` step instead of `wdTagging` + `mediaRating`,
gated by the Flipt flag `image-ingestion-image-scanning` (per image id,
off by default). Flag off, the submitted workflow is unchanged.
- `/api/webhooks/image-scan-result` stays the only callback. It fetches
the workflow and routes on its step types, so both shapes can be in
flight across a flag flip.
- Stages both pipelines share move unchanged to `image-scan-pipeline.ts`.
The legacy service keeps its own parsing and flow.
- New `image-scanning-result.service.ts` reads the imageScanning output
directly (images and video frames), keeps only general tags, and
records csam without acting on it, as legacy does.
- The new pipeline logs to Axiom as `image-scanning-result` /
`image-scanning-ingestion`, submits under
`image_scan_submitted_total{lane="imageScanning"}`, and writes scanner
audit rows as version '2'.
- Remove the non-orchestrator scanner path: the webhook's legacy body
handling, the `IMAGE_SCANNER_NEW` Redis toggle, `ingestImageBulk`,
`image.ingestArticleImages`, `/api/webhooks/reingest-images`,
`/api/internal/add-missing-phash`, `/api/mod/scan-images`, and the
`IMAGE_SCANNING_ENDPOINT` / `IMAGE_SCANNING_MODEL` env vars.
- Bump `@civitai/orchestration-client` to 0.2.0-beta.106 for the
imageScanning types.
Keep the flag off until a deploy has fully rolled out: pods on the
previous build cannot read imageScanning callbacks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
12 KiB
Moderation query port checklist — main app → @civitai/db-queries
Scope: the main app's (this repo) moderation DB queries that should move into the shared package. The moderator app's own queries are already ported (19 domains). This list is what remains, split into Convergence (main-app twin of an already-ported domain — replace the local implementation with the shared one) and Net-new (moderation the moderator app doesn't cover — new domains for the package).
Only
report.service.tscurrently consumes the package (setReportStatusMany). Everything below is still a main-app-local implementation. Many "convergence" items are signature-divergent: the main-app functions do multi-step orchestration (index/cache/appeal/notify side-effects) on top of what the package exposes as narrow primitives — convergence means recompose main-app orchestration on the package primitives, not a 1:1 swap. Keep pure queries in the package; side-effects stay in the app (same rule used for the moderator port).
Part A — Convergence (twins of already-ported domains)
reports (services/report.service.ts)
getReports— moderation queue (twin of packagegetReports)getReportById/getReportByIdsupdateReportById(notes/status → packageupdateReportNotes/setReportStatus)updateImageReportStatusByReason(image.service.ts)createReport— not yet in package (Report + per-type join + ImageRatingRequest/tag-vote/article branches)bulkSetReportStatus— already delegatessetReportStatusMany
image-review (services/image.service.ts)
getImageModerationReviewQueue(needsReview / tagReview / reportReview CTEs)getImageModerationCounts(per-bucket + reported)getImagesModRules/bustImageModRulesCache(→getModerationRuleDefinitions)getModeratorPOITags— POI tag rollup (no exact package twin; near-net-new)
image-moderation (services/image.service.ts, recompose on package primitives)
handleBlockImages→setImageBlocked(+ phash/notify/mod-activity effects stay in app)handleUnblockImages→setImageAccepted(+ appeal/index/tag effects in app)moderateImages(dispatcher)updateNsfwLevel→recomputeImageNsfwLevelupdateImageNsfwLevel(6779) →setImageNsfwLevelupdateImageAcceptableMinor,toggleImageFlag/updateImagesFlagreportCsamImages(image branch; report branch → reports)resolveEntityAppealimage branch →setImageAppealStatus/-Restored/-Rejected,getPendingImageAppealAppellants
image-moderation-effects (services/image.service.ts)
- phash CH writes
addBlockedImage/bulkAddBlockedImages/bulkRemoveBlockedImages(ClickHouse — likely stays app-side) - search-index/cache fan-out helpers (
queueImageSearchIndexUpdate) — effect helpers only
image-tags (services/tag.service.ts, services/image-review.service.ts)
getTagsForReview→getImageTagReviewQueue/getImageTagReviewTagsmoderateTags,disableTags(tag-disable core)createImageTagsForReview/getImagTagsForReviewByImageIds/deleteImagTagsForReviewByImageIds
tags-on-image (services/tagsOnImageNew.service.ts)
insertTagsOnImageNew/upsertTagsOnImageNew/deleteTagsOnImageNew(SP writes →upsert_tag_on_image)applyTagRules/getTagRules
blocklist (services/blocklist.service.ts)
upsertBlocklist,getBlocklistDTO/getBlocklistData,removeBlocklistItems- enforcement reads
throwOnBlockedLinkDomain,throwOnBlockedMessagePattern,getBlockedEmailDomains,stripBenignPhrases(note:utils/moderation-utils.tsblocklists are Redis-backed, not DB — not port targets)
ingestion (services/image.service.ts, jobs/image-ingestion.ts)
getImagesPendingIngestion,getIngestionErrorImages,resolveIngestionError,getIngestionResultsingestImagesjob read/writes (JobQueue-driven)
scanner (services/scanner-audit.service.ts)
recordImageScan→scanner_label_results(→upsertScannerLabelVerdict/insertScannerContentSnapshot)
image-rating-review (services/image.service.ts)
getImageRatingRequests(twin of packagegetImageRatingRequests)
articles (services/article.service.ts, nsfwLevels.service.ts)
unpublishArticleById,restoreArticleById,getModeratorArticles,rescanArticlerecomputeArticleIngestion/recomputeArticleIngestionInTxupdateArticleNsfwLevelsarticleModerationAdapter(EntityModeration pipeline hooks — see net-new #5)
article-rating-review (services/article.service.ts, services/article-rating-review.helpers.ts)
getArticleRatingReviews,getArticleRatingReviewCounts,resolveArticleRatingReviewcreateArticleRatingReview,getArticleRatingReviewForOwner- helpers:
computeArticleDerivedNsfwLevel,evaluateAutoApproveGate,autoResolveArticleRatingReview,maybeAutoResolveDisputeAfterScan
model3d (services/model3d.service.ts, nsfwLevels.service.ts)
unpublishModel3D,deleteModel3D,restoreModel3D,setModel3DNsfwLevel,toggleModel3DFlagupdateModel3DNsfwLevels/updateModel3DNsfwLevelForThumbnailImage- scope check:
model3d-review.service.ts(user reviews) /model3d-report.service.ts(user reports) — confirm whether the ported "model3d" domain means these
comics (routers/comics.router.ts — inline, needs extraction to service fns first; nsfwLevels.service.ts)
setTosViolation,setProjectNsfwLevel,setChapterNsfwLevel,getModReviewQueue,moderatorUnpublishChapterupdateComicChapterNsfwLevels/updateComicProjectNsfwLevels/updateComicNsfwLevels(ForImage)
cosmetics (services/cosmetic.service.ts)
grantCosmeticsToUsers+grantCosmetics→ packagegrantCosmeticsToUsers/insertUserCosmeticGrantgetPaginatedCosmetics(already ported — replace caller)
mod-activity (services/moderator.service.ts)
trackModActivity→ packagerecordModActivity(callers in report/image services + entity-moderation job)
rewards (partial) (services/rewards-bonus-event.service.ts)
getGlobalRewardsBonus— ported- extend:
getActiveRewardsBonusEvent+RewardsBonusEventCRUD (see net-new #12)
sidebar-counts (partial) — package has report/appeal/tag-review counts; main-app dashboard counts differ
- reconcile main-app
getImageModerationCounts/ CSAMgetCsamReportStats/ appealgetAppealCountagainst package counts
Part B — Net-new moderation (no moderator-app equivalent)
1. model / modelVersion moderation (services/model.service.ts, model-version.service.ts)
unpublishModelById,unpublishModelVersionById,deleteModelById,restoreModelById,permaDeleteModelByIdtoggleLockModel,toggleLockComments,setModelsCategory- poi/nsfw/minor/sfwOnly +
lockedPropertiesviaupsertModel/updateModelById getTrainingModelsForModeratorsupdateModelNsfwLevels/updateModelVersionNsfwLevels(nsfwLevels.service.ts)
2. model-flag auto-scan queue (services/model-flag.service.ts, model-file-scan.service.ts)
upsertModelFlag,getFlaggedModels,resolveFlaggedModel,unpublishBlockedModel
3. collection moderation (services/collection.service.ts, nsfwLevels.service.ts)
updateCollectionItemsStatus,setCollectionItemNsfwLevel,updateCollectionsNsfwLevels
4. post moderation (services/post.service.ts, nsfwLevels.service.ts)
deletePost(mod path),updatePostNsfwLevel/updatePostNsfwLevels
5. entity / text moderation (XGuard / Clavata) (services/entity-moderation.service.ts, jobs/entity-moderation.ts)
upsertEntityModerationPending,recordEntityModerationSuccess,recordEntityModerationFailure,getEntityModerationWithImageNsfwLevel- job orchestration:
runModQueue,runModChat,clearAutomatedReports,autoMuteIfScamAccount(+ReportAutomated, JobQueue) - note: main app already has a
ModerationAdapterregistry (moderation-adapters.ts) — decide if the package standardizes on it
6. appeal lifecycle (beyond image status) (services/report.service.ts)
createEntityAppeal,getRecentAppealsByUserId,getAppealCount,getAppealDetails,resolveEntityAppeal(non-image branches)
7. user moderation / enforcement (services/user.service.ts, jobs/confirm-mutes.ts)
toggleBan,toggleContestBan,setUserMuted,setUserModerator,updateUserByIdsoftDeleteUser(CSAM),deleteUser,removeAllContent(~15-table wipe)bulkUnpublishModelsForBannedUser,setLeaderboardEligibilitygetUsers(mod lookup w/ banned/muted/deleted status; raw prefix search) — vs packagesearchUsers(verify Meili vs SQL)confirmMutesjob
8. strike / punishment system (services/strike.service.ts) — entirely net-new, raw-SQL-heavy
- reads:
shouldRateLimitStrike,getActiveStrikePoints,getStrikeSummary,getStrikesForUser,getStrikeHistoryForMod,getStrikesForMod,getUserStandings(dynamic SQL) - writes:
evaluateStrikeEscalation(FOR UPDATE txn),createStrike,voidStrike,expireStrikes,processTimedUnmutes
9. user restriction (generation enforcement) (routers/user-restriction.router.ts, orchestrator/promptAuditing.ts)
getAll(mod queue),resolve(uphold/overturn),submitContext(PromptAllowlist upsert),backfillTriggersauditPromptServer(blocked-prompt → auto-mute + UserRestriction),getCachedPromptAllowlist/bustPromptAllowlistCache
10. CSAM / NCMEC (services/csam.service.ts + csam.service-new.ts) — reconcile the two impls first
createCsamReport,createExternalCsamReport,uploadExternalCsamEvidencegetCsamReportsPaged,getCsamReportStats,getCsamsToReport/getCsamsToArchive/getCsamsToRemoveContentprocessCsamReport,archiveCsamDataForReport,getUserIpInfo(CH+PG),getImageResources
11. cosmetic revoke / target-assign (services/cosmetic.service.ts)
assignCosmeticByTarget(collection/userIds, dry-run),unassignCosmetic,equipCosmeticToEntity/unequipCosmetic
12. RewardsBonusEvent CRUD (services/rewards-bonus-event.service.ts)
getActiveRewardsBonusEvent,upsertRewardsBonusEvent,deleteRewardsBonusEvent,getRewardsBonusEventById,getRewardsBonusEventsPaged
13. knights "down-leveled" review (services/image.service.ts)
getDownleveledImages(ClickHouseknights_new_order_downleveled+ PG join) (image.service.ts) +addToNewOrderQueue(image-scan-pipeline.ts)
14. scan-result ingestion pipeline internals (services/image-scan-pipeline.ts, services/image-scan-result.service.ts)
resolveScanOutcome,auditScanResults,markImageScanError,getAssociatedEntities,evaluateImageModRules,processTags(pipeline);blockImageFromRating,isExemptFromAiVerification(image-scan-result.service)
Sequencing notes
- ClickHouse stays out (per current package charter): phash blocked_images, down-leveled review,
getUserIpInfoCH reads, prohibited-request counts. Port only the PG parts; flag CH like the moderator port did. - Reconcile-before-port: the two CSAM implementations (
csam.service.tsvs-new.ts); Meili-vs-SQL user search. - Extract-before-port: comic moderation lives inline in
comics.router.ts— pull into service fns first. - Highest net-new value: strike system, user ban/mute/delete enforcement, CSAM/NCMEC, entity/text moderation — these are large, raw-SQL-heavy, and have no shared equivalent today.