mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
99a82aa8ca
* feat(app-blocks): approve push-originated requests from Forgejo (Phase 3 core) The load-bearing core of the git-push authoring on-ramp. When a developer git-pushes to civitai-apps/<slug>, the webhook parks a pending publish request with EMPTY bundle pointers (bundleKey=''), but approveRequest called fetchBundleBuffer(bundleKey) unconditionally → it crashed on approve, so a push could be parked but never approved/deployed. approveRequest's bundle source is now source-agnostic: bundleKey → MinIO ZIP (unchanged); else forgejoCommitSha → reconstruct the bundle from the Forgejo repo at the pushed sha; else throw. Everything downstream (platform-owned filter, committed-manifest rewrite, screenshots, commit, sha stamp, build trigger, Phase-2 deploy-state) is unchanged. New `reconstructBundleFromForgejo(slug, ref)` builds a deterministic ZIP from listRepoTreeAtRef + getBlobContent; backfillPublishRequest refactored onto it. Added `listRepoTreeAtRef(slug, ref, org)` (resolves a commit sha via the git trees endpoint; listRepoTree delegates to it, no caller-signature change). No change to the no-trust-on-push deploy gate — pushes still never deploy without mod approval. Tests: push-path approve (reconstructs from Forgejo, no S3 GET, commits, stamps sha, triggers build, deploy_state=building) + ZIP-path regression + reconstruct-helper determinism. Blocks suite 319/319 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(app-blocks): per-developer Forgejo identity + getMyAppRepo (Phase 3 self-service) Lazily provisions each civitai user a scoped, restricted Forgejo identity the first time they request git access to their app, and exposes the clone URL + push credential to the app owner. - Schema: new app_dev_forgejo_identity table (1:1 with userId) storing the Forgejo username + AES-256-GCM-encrypted PAT (keyed on NEXTAUTH_SECRET). Additive migration (manual apply, prod + dev clone). - forgejo.service: createForgejoUser (POST /admin/users, restricted+private, idempotent), getForgejoUser, mintForgejoUserToken (HTTP-Basic as the user — gitea requires the user's own creds; scope write:repository), deleteForgejoUser. - dev-git-access.service: ensureForgejoIdentity — read-or-provision, made CONCURRENCY-SAFE via a DB CLAIM (insert a placeholder row; the userId PK lets one caller own provisioning while the rest wait for the token) — pooler-safe (no advisory locks) and non-destructive. Claim is rolled back on failure. Fixes a race in the first cut where two concurrent first-provisions could purge the user mid-mint and persist a dead token. - blocks.router: getMyAppRepo (protectedProcedure + flag, OWNER-gated) → provisions identity + addCollaborator(write) on this app's repo + returns the authed clone URL + push instructions. Not-approved apps return a "first version is ZIP-only" shape. Isolation: restricted Forgejo user, write only on its own civitai-apps/<slug> repo(s); a push parks a pending review request and CANNOT deploy without mod approval (no-trust-on-push gate unchanged). Tests: forgejo APIs, the claim-first provisioning incl. owner-wait + rollback + orphan edge, owner gate. Blocks suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(app-blocks): "Author via git" UI + getMyAppRepo e2e (Phase 3) Developer-facing surface for the git-push on-ramp + its preview e2e. - AuthorViaGit.tsx: an "Author via git" panel under each approved submission the user owns on /apps/my-submissions. Mount-on-expand — getMyAppRepo is only called when the user clicks (it provisions a Forgejo identity as a side effect, so it must be user-initiated, never on page load; query is staleTime/gcTime 0). Clone URL is credential-MASKED by default (maskCloneUrlCredential, unit-tested) with a reveal toggle; copy buttons copy the real value. Note: "first version is ZIP; new versions via git push; pushes go to mod review, never auto-deploy." Non-owner FORBIDDEN / not-approved are shown as muted states, never a crash. - preview-apps-git-access.spec.ts: e2e (mod fixture) asserting the SECURITY- critical owner-gate live — getMyAppRepo on a non-owned app → FORBIDDEN (throws before any Forgejo user/collaborator is created), unknown id → NOT_FOUND. The full provisioning happy-path (real Forgejo user + git push → park) is unit- covered + a manual preview check (shared-Forgejo state, no safe teardown), mirroring the publish spec's approve exclusion. git-access unit 5/5; e2e discovered under preview-smoke. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(app-blocks): audit follow-ups — wedge recovery, token masking, ban gate (Phase 3) Audit fixes for PR #2587. (The load-bearing approved-sha integrity / no-trust-on- push property was verified SOUND — sha pinned end-to-end, content-addressed.) HIGH — permanent provisioning wedge: a claim row is a standalone insert, so a hard owner crash (pod kill) between the claim and the token write left an empty- token row with no recovery → that user was locked out of git access forever. ensureForgejoIdentity now treats an empty-token claim older than 60s as ABANDONED and atomically reclaims it (optimistic-concurrency guarded on createdAt), then provisions it — no permanent wedge. +test. MEDIUM — the "Steps" block rendered the token-bearing clone URL in cleartext regardless of the reveal toggle (the masker was anchored to a bare URL and didn't touch the embedded URL in the instructions string). maskCloneUrlCredential is now a global mask that handles an embedded credential in any text; the Steps snippet is masked under the same reveal toggle. +test. MEDIUM — getMyAppRepo now refuses to issue a push credential to a banned account (ctx.user.bannedAt → FORBIDDEN). Full revoke-on-ban remains a follow-up. Deferred (documented as follow-ups): orphan-recreate re-grant across the dev's other repos; getMyAppRepo query→mutation; >1000-file tree pagination; Forgejo- user cleanup on GDPR delete. MUST verify before launch: Forgejo restricted-user + private-repo isolation (a dev PAT can't clone other apps / the starter / review org) — config-dependent, not verifiable from the app code. Blocks + git-access suites green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(app-blocks): reclaim guard must be range-based (timestamptz µs vs JS-Date ms) The stale-claim reclaim guarded on exact createdAt equality, but Postgres timestamptz(6) stores microseconds while Prisma reads a millisecond-precision JS Date — the equality would systematically miss, so abandoned claims would never be reclaimed (the wedge fix was inert). Switched to a range guard (createdAt < now-60s), which also serializes concurrent reclaimers via the row lock. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(app-blocks): conditional token-fill/rollback + ban-gate test (re-audit) Second-audit follow-ups (verdict was settled/mergeable; these close the last narrow edge + a test nit). LOW edge: a single owner that stalled >60s between createForgejoUser and the DB write could, after a waiter reclaimed + filled the row, OVERWRITE the winner's good token with its own (now minted against a deleted Forgejo user) → a dead token persisted forever. The fill is now a CONDITIONAL updateMany guarded on the still-empty token (count 0 ⇒ we were superseded ⇒ return the winner's stored token, never persist a dead one). The failure rollback is likewise deleteMany guarded on the empty token, so it can't nuke a reclaimer's filled row. NIT: added a getMyAppRepo test asserting a banned owner → FORBIDDEN with nothing provisioned. dev-git-access + getMyAppRepo suites green (15/15 affected). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
154 lines
7.8 KiB
TypeScript
154 lines
7.8 KiB
TypeScript
import { expect, test } from '@playwright/test';
|
|
import type { APIRequestContext } from '@playwright/test';
|
|
import { storageStatePath } from './preview-fixtures';
|
|
import { trpcQuery } from './preview-trpc';
|
|
|
|
/**
|
|
* Preview-e2e (F / Phase 3 git-push self-service): App Blocks `getMyAppRepo`
|
|
* OWNER-GATE + endpoint wiring. Proves the SECURITY-CRITICAL property — a
|
|
* logged-in NON-OWNER cannot mint a push credential for someone else's app — and
|
|
* that the proc is reachable, WITHOUT polluting the SHARED Forgejo.
|
|
*
|
|
* Runs as the `mod` fixture (id 2000000001, ci-smoke-mod): `getMyAppRepo` is a
|
|
* `protectedProcedure` gated by `features.appBlocks` (the Flipt mod segment), so
|
|
* a non-mod tester would be UNAUTHORIZED/FORBIDDEN before the owner check even
|
|
* runs and the gate under test would be untestable. mod is also rate-limit-exempt
|
|
* and clears the flag. Crucially, the `mod` fixture is NOT the owner of the apps
|
|
* `listAvailable` surfaces (those are real prod-clone apps owned by other users),
|
|
* so calling getMyAppRepo on one exercises the NON-OWNER path → FORBIDDEN.
|
|
*
|
|
* --- THE PROVISIONING HAPPY-PATH IS INTENTIONALLY NOT E2E'd HERE -------------
|
|
* The OWNER happy-path (status='approved' + caller IS the owner) is deliberately
|
|
* NOT covered, for the same reason the publish spec defers approve→build→render
|
|
* and the install spec defers generate/buzz: a successful getMyAppRepo call
|
|
* `ensureForgejoIdentity(userId)` — it CREATES a real `dev-<userId>` user on the
|
|
* SHARED Forgejo instance and grants it `write` collaborator on the app repo
|
|
* (blocks.router.ts getMyAppRepo → dev-git-access.service + forgejo.service). That
|
|
* is durable, side-effecting state on shared infra with NO cheap teardown (needs
|
|
* Forgejo admin), so an automated preview run must never trigger it. The full
|
|
* provisioning path (real Forgejo user + scoped token + git push → parked review
|
|
* request, never auto-deploying) is verified by the UNIT suite (dev-git-access /
|
|
* forgejo / git-push gate tests) + a MANUAL preview check — exactly the Phase-1
|
|
* approve→render exclusion and the publish spec's approve exclusion.
|
|
*
|
|
* This spec therefore covers what IS safely automatable: the owner-gate (a
|
|
* non-owner is rejected BEFORE any Forgejo side effect — the owner check throws
|
|
* first) + the NOT_FOUND wiring for a bogus id. Both reject before
|
|
* ensureForgejoIdentity is ever reached, so neither touches Forgejo.
|
|
*
|
|
* Verified shapes (against the worktree's blocks.router.ts, paths rel. to civitai/src):
|
|
* - blocks.listAvailable (publicProcedure + flag + 60/60 rateLimit; input
|
|
* listAvailableSchema, `{}` valid) → `{ items: AvailableBlock[]; nextCursor? }`
|
|
* (NOT a bare array). Each item's `id` is the appBlockId. Used to discover an
|
|
* id the mod does NOT own (skip-if-empty, annotated). The returned apps are
|
|
* `status='approved'` (the registry filters), which is also what drives
|
|
* getMyAppRepo PAST the not-yet-available short-circuit and INTO the owner gate.
|
|
* - blocks.getMyAppRepo (blocks.router.ts getMyAppRepo, protectedProcedure +
|
|
* enforceAppBlocksFlag; input { appBlockId: string (1..64) }):
|
|
* • caller is NOT the app owner → throws TRPCError FORBIDDEN ('Not the app
|
|
* owner'). The owner check (block.app.userId !== ctx.user.id) runs BEFORE
|
|
* ensureForgejoIdentity, so NO Forgejo user/collaborator is created.
|
|
* • unknown appBlockId → throws NOT_FOUND ('App block not found') via
|
|
* throwNotFoundError, before any owner/Forgejo logic.
|
|
* • (owner + approved → { notYetAvailable:false, cloneUrl, httpUrl,
|
|
* forgejoUsername, instructions, firstVersionIsZip:false } — NOT asserted
|
|
* here; see the exclusion note above.)
|
|
*/
|
|
|
|
const ROLE = 'mod' as const;
|
|
|
|
type AvailableBlock = { id: string };
|
|
type ListAvailableResult = { items: AvailableBlock[]; nextCursor?: string };
|
|
|
|
// The tRPC v11 (superjson) error envelope for a batched GET:
|
|
// [{ error: { json: { message, code (numeric), data: { code: string,
|
|
// httpStatus } } } }]
|
|
// We read the human-readable `data.code` ('FORBIDDEN' / 'NOT_FOUND').
|
|
type TrpcErrorEnvelope = {
|
|
error?: { json?: { message?: string; data?: { code?: string; httpStatus?: number } } };
|
|
};
|
|
|
|
/**
|
|
* Raw batched-GET call to a tRPC query that we EXPECT to error, returning the
|
|
* parsed error code + message. The shared `trpcQuery` helper throws on a tRPC
|
|
* error (good for happy-path), but to assert the ERROR CODE deterministically we
|
|
* inspect the envelope ourselves. Mirrors preview-trpc's batched wire format +
|
|
* CSRF (Origin/Referer) stamping.
|
|
*/
|
|
async function trpcQueryExpectError(
|
|
request: APIRequestContext,
|
|
proc: string,
|
|
input: unknown,
|
|
previewUrl: string
|
|
): Promise<{ code: string | undefined; message: string | undefined; httpStatus: number }> {
|
|
const enc = encodeURIComponent(JSON.stringify({ '0': { json: input } }));
|
|
const res = await request.get(`/api/trpc/${proc}?batch=1&input=${enc}`, {
|
|
headers: { origin: previewUrl, referer: `${previewUrl}/` },
|
|
});
|
|
const body = (await res.json().catch(() => ({}))) as unknown;
|
|
const entry = (Array.isArray(body) ? body[0] : body) as TrpcErrorEnvelope;
|
|
return {
|
|
code: entry?.error?.json?.data?.code,
|
|
message: entry?.error?.json?.message,
|
|
httpStatus: res.status(),
|
|
};
|
|
}
|
|
|
|
test.describe('App Blocks getMyAppRepo owner-gate + wiring (mod, no Forgejo side effects)', () => {
|
|
test.use({ storageState: storageStatePath(ROLE) });
|
|
|
|
test('non-owner → FORBIDDEN; unknown id → NOT_FOUND (no credential minted)', async ({
|
|
page,
|
|
}) => {
|
|
const previewUrl = process.env.PREVIEW_URL ?? '';
|
|
|
|
// Warm the request context against the preview origin so page.request shares
|
|
// the mod auth cookie + a navigated origin (the CSRF gate needs Origin/Referer
|
|
// host allowlisted; NEXTAUTH_URL == the preview URL). domcontentloaded ONLY —
|
|
// never networkidle.
|
|
await page.goto('/', { waitUntil: 'domcontentloaded' });
|
|
const request = page.request;
|
|
|
|
// DISCOVER an approved appBlockId the mod does NOT own (never hardcode — the
|
|
// weekly prod clone's approved set varies, and could be empty → skip).
|
|
const listing = await trpcQuery<ListAvailableResult>(request, 'blocks.listAvailable', {});
|
|
const blocks = listing?.items ?? [];
|
|
test.skip(
|
|
blocks.length === 0,
|
|
'No approved app blocks in this dev-DB clone — nothing to probe the owner-gate against (the weekly prod clone can have zero). Skipping rather than hard-failing.'
|
|
);
|
|
const appBlockId = blocks[0].id;
|
|
expect(typeof appBlockId, 'discovered appBlockId should be a string').toBe('string');
|
|
|
|
// OWNER-GATE: the mod fixture is NOT the owner of this prod-clone app, so
|
|
// getMyAppRepo must reject with FORBIDDEN. This is the security-critical
|
|
// assertion: a logged-in non-owner is denied a push credential. The owner
|
|
// check throws BEFORE ensureForgejoIdentity, so this creates NO Forgejo user
|
|
// and grants NO collaborator — safe to run automated against shared infra.
|
|
const forbidden = await trpcQueryExpectError(
|
|
request,
|
|
'blocks.getMyAppRepo',
|
|
{ appBlockId },
|
|
previewUrl
|
|
);
|
|
expect(
|
|
forbidden.code,
|
|
`getMyAppRepo on an app the mod does not own should be FORBIDDEN (was: ${forbidden.code} / "${forbidden.message}")`
|
|
).toBe('FORBIDDEN');
|
|
|
|
// WIRING: an unknown appBlockId resolves to no AppBlock row → NOT_FOUND,
|
|
// thrown before any owner/Forgejo logic. A non-existent-but-charset-valid id.
|
|
const bogusId = `ci-smoke-nope-${Date.now()}`.slice(0, 64);
|
|
const notFound = await trpcQueryExpectError(
|
|
request,
|
|
'blocks.getMyAppRepo',
|
|
{ appBlockId: bogusId },
|
|
previewUrl
|
|
);
|
|
expect(
|
|
notFound.code,
|
|
`getMyAppRepo on a non-existent appBlockId should be NOT_FOUND (was: ${notFound.code} / "${notFound.message}")`
|
|
).toBe('NOT_FOUND');
|
|
});
|
|
});
|