mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
Zachary Lowden
b0711ee568
* feat(apps): let an App Block publish a real post from its own outputs
Adds the host half of a new App Blocks capability: a block can ask the host
to create a REAL, published Post on the viewer's profile, built from the
app's own generation outputs and/or images it previously published, with an
optional model-version gallery attach.
This is a SIBLING of the existing shared-grid publish bridge, not an
extension of it. Two reasons decided that:
* the existing publish payload carries a single required workflow id and
cannot express a post built from several sources;
* its consent copy says the images "become visible to other viewers of
this app", which is false for a profile post — and that sentence is the
security control, not decoration.
New scope `posts:write:self`, mapped to the existing "upload media & create
posts" OAuth bit. It is SENSITIVE (a manifest declaring it must justify it)
and CONSENT-PROMPTED (deliberately not consent-exempt), so a token carries
it only after the viewer grants it. Wired end to end: registry, sensitive
set, runtime binding, consent description, canonical manifest schema, and
both dev-mint allowlists. It is withheld from BOTH moderator-review mint
allowlists — a mod previewing an unapproved third-party app must never
publish public content under their own name.
Two procedures, because the consent dialog has to be trustworthy:
`previewPostFromApp` resolves, server-side, everything the confirm renders
(the exact copy, the tag names that will actually apply, host-fetched model
and version names, real thumbnails); `createPostFromApp` re-runs every guard
and writes. The block is sandboxed and cannot be trusted to display
truthfully, so the dialog asserts what the SERVER resolved, never what the
block sent. The preview confers no authority.
Controls, each with a negative test:
* SELF-DEALING — refuses a gallery attach whose model is owned by the
calling app's own publisher. This is the one control that removes the
payoff of routing viewers' posts at your own models; everything else
only raises the cost. A colluding second account still defeats it, which
is what the attribution marker below is for.
* The gallery gate is otherwise strictly stricter than the native path,
which checks nothing at all: published + public + undeleted, or refused.
* A block may NEVER mint a site tag. Requested names resolve against
existing tags only; unmatched names are dropped and shown in the confirm.
Moderation, system and admin-only tags are excluded even when they match.
* Server-side text bounds and a link refusal on title and detail, plus
blocked-content screening over title, detail AND the resolved tags (the
native path screens only the first two).
* Server-authoritative attribution in post metadata, using the same key
and semantics as the image-side marker so one moderation sweep reads
both. No post input schema exposes that field, so no client can forge or
suppress it, and the badge must render from the column rather than from
block-supplied copy. An index ships with it rather than after an incident.
* A DEDICATED rate bucket for posts, separate from the image-weighted
publish bucket; the per-image origin cost is still charged to the latter
so this path cannot be used to bypass it.
* A durable audit row for every outcome once the request is admitted,
with a named Activity-feed sentence rather than the generic fallback.
* Its own kill switch, independent of the App Blocks runtime flag, so
widening that flag toward GA does not arm public post creation on the
same day. SHIPS OFF: the flag does not exist yet, so an absent flag
resolves false for everyone and the whole capability is dark as merged.
* The confirm's exactly-once reply latch is the FIXED shape, so "declined"
means no post exists even if the dialog is dismissed mid-write.
* The write refuses if the resolved image set no longer matches the count
the viewer confirmed — the one divergence every authorization check
would happily wave through.
The write is atomic: create, adopt images, publish in one transaction. Image
adoption is a bounded update whose matched count IS the ownership and
provenance proof, so a row that changes underneath us aborts the whole thing
instead of leaving a public artefact nobody agreed to.
Known consequence, documented for app authors: adopting a previously
published image into a post removes it from the app's own shared grid, since
that read is scoped to post-less rows. An app cannot both keep an image in
its grid and let the viewer post it.
The min-trust gate moved to its own module now that it has a second caller;
the rule, its signals and its exact messages are unchanged.
The block-side message pair and the CLI manifest mirror are co-requisites
that ship separately and AFTER this deploys, since their drift guards
compare against the live published schema.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(apps): declare the two new post procs in every host-rendering trpc mock
23 suites render PageBlockHost against a hand-listed mock of the tRPC client —
one entry per procedure the component reads. The component now reads two more,
so every one of those mocks returned `undefined` for them and the whole
component threw at render.
🔴 THE FAILURE DOES NOT LOOK LIKE A MISSING MOCK. The DOM comes back EMPTY, so
each assertion fails with "no element with data-testid=…" — which reads as the
element having been removed, in files that measure layout and have nothing to
do with posting. The geometry suite's own POSITIVE CONTROL failed too, and that
fixture is hand-built markup that never touches the new code; a reader would
reasonably conclude the harness was broken rather than the mock incomplete.
Caught by the geometry project, not by the new tests: every test added with the
feature mocks the service layer directly and none of them render the host, so
the seam between "the component reads a procedure" and "the mock declares it"
was owned by no suite that changed. That is the gap, not the fix.
Also adds a typed constant for the refusal codes the HOST itself emits on this
bridge, so a block can branch on them and a typo is a compile error. It is
explicitly NOT an allowlist: the reply's error field must stay shape-checked,
because server messages travel through the same field and a reply that fails
validation is dropped before correlation — wedging the block for ten minutes
instead of showing the reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(apps): name the scan-timing constraint on posting an already-published image
An image published through the grid bridge is not immediately postable: that
bridge returns ids before any scan runs, and the adopt path requires a terminal
scan. An app that publishes and posts in one breath gets a refusal that reads
as a bug rather than as a wait.
* docs(apps): add posts:write:self to the block-scope table, and mark the table non-exhaustive
The table was already missing the shared-storage and collections scopes.
Adding one row while leaving four absent would make it look complete, so it
now says plainly that the constant is the authority and names what is missing.
* fix(apps): app-level post ceiling, refuse a non-terminal workflow, drop a dead re-export and three dangling citations
Four round-0 audit fixes on the app-post path.
1. AGGREGATION GAP — add an APP-scoped post ceiling.
checkBlockPostRateLimit keys on blockInstanceId, which bounds ONE install, so
an app with N installs gets N times the ceiling and nothing sees the total.
Adds checkBlockPostAppRateLimit(appId) on its own ':post-app:' sub-namespace,
checked alongside (not instead of) the per-instance bucket. 300/hour.
The number is NOT data-derived and the code says so: it is 100x the
per-instance ceiling, i.e. 100 distinct installs each at their own hourly max
in the same hour, and it is a starting value to be revised from the
block_scope_invocations audit rows. Sized loose on purpose — a too-tight
aggregate throttles a popular legitimate app and reaches users as "posting is
broken", which is worse and quieter than the abuse it prevents. The
per-instance 3/hour is unchanged and still labelled a guess.
Fails open on a Redis error like every sibling limiter, stated at the call
site so nobody reads either bucket as a hard cap.
2. PREVIEW/WRITE DIVERGENCE — refuse a non-terminal workflow source.
The generator of the race is a workflow that can still gain an output between
the two phases. projectAppWorkflow already computes status and
resolveOwnedWorkflowOutputs was discarding it, so the refusal costs zero extra
IO and binds every caller. pending/processing are refused with BAD_REQUEST;
succeeded/failed/expired/canceled are admitted because the output set is
frozen in all four. An unrecognised upstream status fails closed. The terminal
list is bound to the wire union with satisfies, so a status rename breaks the
build rather than widening the gate.
confirmedImageCount is KEPT and made REQUIRED, not deleted: the terminality
gate removes one known race, the count pins the whole resolved set, so a
future source arm or an expiring blob still surfaces as a refusal. An
integrity check a caller may decline is not an invariant. It moves out of the
shared preview/write payload shape — it is the write's echo of what the
preview returned, so the preview cannot be asked for it.
3. DEAD RE-EXPORT + a comment that misstated a dependency. The re-export claimed
to keep "every existing importer" working; the population is empty. Every
importer of apps-shared.router takes appsSharedRouter/appsModRouter,
sanitizeDiscordText, or the counter helpers — none takes assertSharedWriteTrust,
MIN_ACCOUNT_AGE_MS or REQUIRE_PAID_TIER. Removed, with the enumeration recorded
in place of the false claim.
4. THREE CITATIONS POINTING NOWHERE — "operator decision 3", "operator decision 4"
and "the design note calling for Scanned on every image" referenced a document
that is not in the tree. Inlined the substance at each site, written as
decisions with the rejected alternative named, and dropped the pointers. No new
doc: a pointer to a doc that can rot is what produced this.
Tests: app-ceiling refusal plus a different-app-unaffected case (the half that
proves app-scoping rather than a global key); the full pending/processing/
succeeded/failed/expired/canceled matrix plus an unrecognised status; ordering
against both ownership proofs; required-vs-optional on the confirm count.
* fix(apps): an app-created post was invisible to everyone but its author
Both `Post` triggers this path depends on are declared `AFTER UPDATE OF
"publishedAt"` — INSERT is in neither event list — and `writeBlockPost`
writes `publishedAt` inside the `post.create`. So neither fired, and
nothing else on this path writes `Post.nsfwLevel`.
The post therefore kept its schema default `nsfwLevel = 0` forever, and
both non-owner reads gate on it: `getPostDetail` admits a non-owner only
on `{ publishedAt: { lt: now }, nsfwLevel: { not: 0 } }`, and
`getPostsInfinite` masks on `(p."nsfwLevel" & browsingLevel) != 0`. A
post whose images are all `published` sources was a permanent 404 for
everyone but its author and absent from the profile tab and every feed,
while its images stayed visible in galleries — which reads as a cache
bug. Only that arm was broken: a post containing a fresh output recovers
by accident, because the output's later scan fires the Image trigger and
the job bit_ors over every image of the post.
`applyBlockPostPublishEffects` now re-issues both triggers. It ENQUEUES
`JobQueue(Post, UpdateNsfwLevel)` rather than calling
`updatePostNsfwLevels` directly, because the enqueue is what the trigger
does, so the consumer behaves natively and walks the post to the model
version it is attached to. Same fix seeds the `PostMetric(AllTime)` row
the metrics trigger would have created, so `ageGroup` is not left NULL.
Also in this pass:
- `droppedTags` reached the consent dialog unsanitised. It is the
block's own tag strings echoed back, so a block could put bidi
overrides and zero-width padding into the host surface that IS the
security control. Stripped at the source in tag normalisation,
re-sanitized at the render point, and included in the blocklist
screen. The module doctrine comment claimed the app name was the only
block-influenced value; corrected.
- `resolveOwnedWorkflowOutputs` claimed to return the same ordered
projection the block saw and did not — it filtered the host
allowlist, which silently renumbered later outputs, so an index could
select a different image than the one it named. It now blanks the slot
in place and the selection site refuses a blanked index.
- `applyBlockPostPublishEffects` had no direct test; its only coverage
mocked it wholesale. Added a ledger test asserting the exact effect
set, failing when it grows or shrinks, plus a test that drives the
visibility symptom end to end rather than asserting a call.
- ClickHouse `nsfw` was hardcoded false for every app-created post. It
is now derived from the adopted images, which is what the level the
job queue computes will be built from.
- Two comments corrected: the write-trust module claimed a re-export
that the other file explicitly denies, and the shared post preamble
claimed preview and write can never drift when write-trust is
deliberately write-only.
* fix(apps): make the nsfw-level enqueue atomic with the publish, and correct the trigger ledger
The enqueue that keeps an app-created post visible was issued from
applyBlockPostPublishEffects — a function the router calls AFTER
writeBlockPost's transaction has committed, and whose every rejection it
swallows into a log line. So a connection reset, pool exhaustion or
statement timeout on that single INSERT produced a committed post with
nsfwLevel = 0 forever: a permanent 404 to non-owners, absent from the
profile tab and every feed, with no reconciliation sweep (the temp
backfill covers ModelVersion/Model only) and no alert. The trigger it
stands in for fires INSIDE the publishing transaction, so the parity was
never exact.
It is now issued on the transaction client, as the trigger's own
statement, ON CONFLICT DO NOTHING included. The effects function's
docblock says why nothing visibility-deciding may live there: every
effect it issues must survive being silently dropped.
Tested at the symptom rather than the call: the publish ALONE, with the
post-commit path never invoked, must still leave the post readable by the
two non-owner predicates; and it must still do so when every post-commit
statement fails and the router swallows it. A "was enqueueJobs called"
assertion passes in both designs, which is why it was replaced. Plus the
structural half — the insert lands on the transaction client, not the
global one — and the rollback arm.
Also in this pass:
- The trigger enumeration claimed TWO Post triggers and that INSERT was
in neither event list. There are FOUR, and one of them is in a
migration rather than programmability, so a sweep of programmability
alone gets this wrong. post_published_at_change is a third UPDATE-only
one; it is not re-issued because its effect already happens by accident
— image_sort_at_before fires on the adopt, which runs after post.create
in the same transaction. That coverage is incidental and fragile, so
its ordering half is now pinned by a test and its SQL half is marked
unverified. The fourth, trg_moderation_post, DOES fire on INSERT, so
the block-supplied copy is queued for moderation with no help from us.
- A workflow source that named NO indexes over outputs with one
off-allowlist slot refused the entire post. The refusal exists because
skipping an index the block NAMED would publish a different image than
the one named; an omitted imageIndexes names nothing and is documented
as "every available output". It now skips the blanked slot there and
refuses only an explicitly named one.
- That made the all-blanked guard load-bearing rather than redundant, and
it is now pinned: deleting it changes the refusal to two different
messages that point an author at the wrong problem. Measured before:
deleting it left the file green, because the mutant died to the
downstream guard's identical message.
- The ClickHouse nsfw flag reduced over published members only, so a
MIXED post reported SFW permanently even after its fresh output scanned
X. A fresh member now unrates the whole post, matching the all-fresh
arm's conservative direction.
- Two claims narrowed to what they actually cover: the effect ledger
records only calls through the mocked modules, and the Tag.name
no-format-characters premise is an unverified empirical assumption
whose character class includes ZWJ/ZWNJ.
* docs(apps): correct five round-3 audit findings in the post-from-app comments
Comment and test-assertion corrections only. No behaviour change: the diffs in
both source files are comment-only, verified mechanically.
1. The SQL offered for settling the Tag.name premise did not run. PostgreSQL's
regex engine has no \p{...} property classes, so the suggested
`name ~ '[\p{Cf}]'` fails with "invalid regular expression: invalid escape \
sequence" — leaving an operator with a syntax error and the assumption still
unverified, which is the one thing that note exists to prevent. Replaced with
an explicit enumeration of the full Cf set written in ARE \uXXXX / \UXXXXXXXX
escapes, so the query also carries no invisible characters of its own, plus a
line saying why it enumerates so nobody simplifies it back. Verified on a
throwaway PostgreSQL 17.10 with both controls. The \p{Cf} in the code is
JavaScript's engine and is correct; it is untouched.
2. The cap docblock claimed a refusal rather than a truncation unconditionally.
The maxCount +1 trick is what makes an over-cap request visible, but the skip
arm for an unnamed index on a blanked slot consumes a selected slot without
contributing, absorbing that headroom. Docblock corrected to state the
exception; behaviour deliberately left alone, since the preview runs the same
resolver so the published set still matches the consented thumbnails.
3. The ON CONFLICT assertion targeted calls[0] positionally, two lines after the
same test computed a content-based selector. Now selects the statement by its
INSERT INTO "JobQueue" token, with a length check as a positive control on the
selector itself.
4. "byte-for-byte the one create_job_queue_record runs" was false — the trigger's
VALUES clause has no ::integer and different value sources. The statement is
character-identical to enqueueJobs' per-row SQL; cited that instead, and
stated the parity that actually matters (identical bare conflict target
against the JobQueue primary key).
5. The trigger ledger is complete for this repo, and the repo is not a complete
record. bitdex_post_54f0a619 appears only as a DROP on "Post", with no CREATE
anywhere in the tree, so this table has already carried a trigger the
enumeration method structurally could not see. Named pg_trigger on the
production database as the authority and gave the query, so "re-derive" is
actionable.
Also fixed a duplicated clause left by an incomplete edit in the
publish_post_metrics_trigger comment.
Gates: typecheck 0 errors; test:lint-rules 38 files / 530 tests; the post,
blocks.router and AppBlocks suites 71 files / 1450 tests; prettier clean (both
checks run against a validated instrument).
Mutation for 3: deleting ON CONFLICT DO NOTHING from the JobQueue insert fails
the new assertion at its own line with its own message, after the positive
control passes. Adding an earlier raw statement that also carries the phrase
while stripping it from the JobQueue insert PASSES 12/12 under the old positional
assertion and FAILS under the new one — the regression this change prevents,
demonstrated.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>