BlobData refactor

This commit is contained in:
Briant Diehl
2026-04-15 17:28:15 -06:00
parent fefa1e2054
commit 06928c5b27
26 changed files with 1288 additions and 788 deletions
+157
View File
@@ -0,0 +1,157 @@
# BlobData Refactor — Follow-ups
Context: we converted `BlobData` from a single concrete class into an abstract base with three concrete subclasses (`ImageBlob`, `VideoBlob`, `AudioBlob`), replaced per-blob `status` with `available` + `step.status`, and renamed `images``output` at the class/wire layer. The items below are the loose ends that weren't in scope for that pass.
Primary files:
- [src/shared/orchestrator/workflow-data.ts](../src/shared/orchestrator/workflow-data.ts)
- [src/server/services/orchestrator/orchestration-new.service.ts](../src/server/services/orchestrator/orchestration-new.service.ts)
---
## 1. Generic `BlobData<T>` for subclass props
**Where:** [workflow-data.ts](../src/shared/orchestrator/workflow-data.ts) — abstract `BlobData` class + `ImageBlob` / `VideoBlob` / `AudioBlob`.
**What:** Each subclass currently re-declares its extra fields (`width`/`height`/`aspect` on Image and Video; `duration` on Audio). Explore whether a generic `BlobData<TExtra>` where `TExtra` describes the subclass-only shape would eliminate the per-subclass boilerplate.
**Sketch:**
```ts
abstract class BlobData<TExtra = {}> {
readonly type!: 'image' | 'video' | 'audio';
url!: string;
// ...shared fields
}
class ImageBlob extends BlobData<{ width: number; height: number; aspect: number; previewUrl?: string | null; previewUrlExpiresAt?: string | null }> {
readonly type = 'image' as const;
}
```
**Tradeoffs to settle:**
- TypeScript won't auto-promote `TExtra` into instance properties — you'd need `implements` on the subclass or a mapped-type merge, and `Object.assign(this, data)` still does the runtime work. May not actually reduce boilerplate.
- Harder for callers to read the class at a glance (fields are in the type parameter, not the class body).
- Worth prototyping before committing.
---
## 2. ~~`BlobData.workflow` non-null assertion~~ — **Resolved**
`StepData` constructor now takes `workflow: WorkflowData` as a required parameter (was `wfMetadata?`, `workflow?` previously). `StepData.#workflow` is non-optional, `StepData.workflow` returns `WorkflowData` (not `| undefined`), and `BlobData.workflow` no longer needs `!` — the invariant "every blob has a workflow" is now enforced at the type level.
Notes:
- `_setWorkflow` escape hatch kept for the rebuild path in `WorkflowData` ctor (existing StepData instances can be re-parented onto a new WorkflowData during immer-style updates).
- Tests in [workflow-metadata.test.ts](../src/server/services/orchestrator/__tests__/workflow-metadata.test.ts) updated to construct a bare `WorkflowData` via a `makeWorkflowData()` helper.
---
## 3. `AudioBlob` aspect default vs. `OutputBlob` intermediate
**Where:** [workflow-data.ts — `AudioBlob`](../src/shared/orchestrator/workflow-data.ts)
Currently `AudioBlob` has `readonly aspect = 1` as a convenience for consumers that generically read `.aspect` across the blob union. The original thought was to introduce an `OutputBlob` intermediate class between `BlobData` and `ImageBlob`/`VideoBlob`/`AudioBlob` that holds `aspect` with a default of 1.
**Decide:**
- Keep `aspect = 1` on `AudioBlob` only (current state, minimal).
- Or hoist to an intermediate class (`OutputBlob` or similar) with a default of 1, which audio inherits and image/video override with real values.
Low priority — current state works. Revisit only if another cross-type property appears with a similar "sensible default for one variant" pattern.
---
## 4. ~~`step.metadata.images` wire key rename~~ — **Resolved (dual-key)**
Landed approach (none of the originally proposed options):
- **Server passes both keys raw.** [`NormalizedStepMetadata`](../src/server/services/orchestrator/orchestration-new.service.ts) declares both `output` (current) and `images` (legacy). `formatStep` emits whatever the orchestrator stored under each key — no merging, no migration. New writes land under `output`; legacy state stays under `images`.
- **Client merges for display.** [`BlobData.outputMeta`](../src/shared/orchestrator/workflow-data.ts) returns `{ ...legacy, ...current }` per-blob — current key wins per-field, legacy fields not yet rewritten still show through.
- **Client writes `output/*`.** [`generationRequestHooks.ts`](../src/components/ImageGeneration/utils/generationRequestHooks.ts)'s jsonPatch builder targets `output/*`. The orchestrator's strict ASP.NET typed jsonpatch accepts these (the DTO has both `output` and `images` properties), and rejects writes to a path whose parent doesn't yet exist — so the client emits init ops (`add path:'output' value:{}`, `add path:'output/${id}' value:{}`) when the **raw** `output` is missing on the orchestrator. Decision is based on raw `output`, not merged.
- **Optimistic update + tag sync use the merged view.** Otherwise the post-patch state would think a legacy-liked workflow has no liked images and would strip the `feedback:liked` tag from the workflow.
- **Filter-cache pruning.** `updateImages` now removes workflows from caches whose filter tags they no longer match (e.g. unliking the last liked image while filtered to "liked").
Why we didn't migrate: data loss was a non-starter, and the orchestrator's typed-patch rejected the bulk migration op anyway. The dual-key path keeps both old and new workflows working without any backend coordination.
Future cleanup (not urgent): a one-shot backfill job to rewrite `metadata.images``metadata.output` on all workflows. Once everything's under `output`, remove the legacy reader (`legacyImages` lookup in `BlobData.outputMeta` and `updateImages`) and drop `NormalizedStepMetadata.images`. No rush — the legacy shim is small, contained, and self-healing in the sense that any post-rename write to a workflow leaves that workflow's `output` key populated for future reads.
---
## 5. `Object.assign(this, data)` type/runtime gap
**Where:** [workflow-data.ts — `BlobData` constructor](../src/shared/orchestrator/workflow-data.ts)
```ts
constructor({ data, ...opts }: BlobConstructorArgs) {
Object.assign(this, data);
// ...
}
```
**Problem:** `Object.assign` is how every blob field gets populated (id, url, width, height, previewUrl, duration, etc.). TypeScript can't see the assignment, which is why the class declares `url!: string`, `id!: string`, `available!: boolean`, etc. with definite-assignment assertions. Implications:
- If `NormalizedImageOutput` adds a field on the wire, the instance silently carries it even without a class declaration — unreachable from typed reads.
- If a subclass declares a field the wire shape doesn't provide, the field is silently `undefined` at runtime — no compile error.
- The class hierarchy and the normalized wire types (`NormalizedImage/Video/AudioOutput`) are two sources of truth kept in sync by hand. `satisfies` annotations in `formatStepOutputs` cover the *emission* half; nothing covers the *consumption* half.
**Options:**
1. **Explicit field copy** — in each subclass ctor, assign only the fields the class declares. Kills the silent-field-drift problem at the cost of ctor boilerplate.
2. **Derive one from the other** — generate the class-field declarations from the wire type (or vice versa) so drift is a compile error. Non-trivial — would likely need a codegen step or a heavy conditional-type utility.
3. **Accept it, add guardrails** — write a runtime test that instantiates each subclass from a fully-populated wire payload and asserts every declared field got a value. Catches drift at test time, not compile time.
4. **Leave it.** Current state. The `!` assertions are honest *today*; any drift becomes a bug for future-us.
**Recommendation:** Start with option 3 (cheap, catches the most common drift case). Promote to option 1 if drift bugs actually materialize.
---
## 6. `_setWorkflow` re-parenting escape hatch
**Where:** [workflow-data.ts — `StepData._setWorkflow`](../src/shared/orchestrator/workflow-data.ts); called from the `WorkflowData` constructor when `rawStep instanceof StepData`.
**Context:** A `WorkflowData` can be re-constructed from a prior `WorkflowData`'s `steps` array (e.g. during immer-style query-cache mutations). When that happens, existing `StepData` instances are reused — but their `#workflow` reference still points at the old `WorkflowData`. `_setWorkflow(this)` rewires them onto the new parent.
**Problem:** `_setWorkflow` is mutation-in-disguise. Consumers holding a reference to a StepData can have its workflow replaced out from under them without notice. The `#private` field makes this invisible at the type level.
**Options:**
1. **Keep it.** Current state. Acceptable because the only caller is `WorkflowData`'s own ctor, and the replacement is always onto a structurally-equivalent workflow (same id, same metadata snapshot). Low risk in practice.
2. **Collapse to always-fresh construction.** Drop `_setWorkflow`; the `WorkflowData` ctor always constructs new `StepData` instances (and therefore new `BlobData` instances) even when the input contains pre-wrapped StepData. Simpler invariant, at the cost of re-wrapping every blob on every workflow mutation. Measure perf first — if cache-update churn is hot, this is expensive.
3. **Immutable reconstruction.** Expose a `StepData.withWorkflow(workflow)` method that returns a *new* StepData sharing the same underlying fields and output array but with a different `#workflow`. The `WorkflowData` ctor calls this instead of mutating. Middle ground — cheap and no hidden mutation.
**Recommendation:** Option 1 until we measure. Option 3 if we ever want to hand StepData references to code that shouldn't see mutations.
---
## 7. `available: false` post-step-success — investigate orchestrator behavior
**Where:** observed in aceStepAudio workflows; see the error-card handling in [GeneratedOutputWrapper.tsx](../src/components/ImageGeneration/GeneratedOutputWrapper.tsx).
**Context:** During the refactor we observed workflows where `step.status === 'succeeded'` but the step's output blob has `available: false`. We surfaced this in the UI as an error card (`BlobData.errored` getter + wrapper render branch), but the underlying cause wasn't investigated.
**Questions:**
- Is `available: false` after step-success a genuine terminal failure (worker produced no output), or a transient "post-processing in progress" state that the orchestrator resolves later?
- If transient, what's the expected window, and should clients poll / re-query instead of showing an error card immediately?
- Does this happen only for `aceStepAudio` (which has an unusual blob lifecycle — the audio/video is assembled after the job reports done), or for other step types too?
**Action:** Open a ticket against the orchestrator team. If transient, we may want to delay the error card (e.g. only show after N seconds of `succeeded + !available`) or drive it off a different signal entirely.
---
## 8. ~~`GeneratedImage.stories.tsx` — rename / realign~~ — **Resolved**
Renamed to [`GeneratedOutput.stories.tsx`](../src/components/ImageGeneration/GeneratedOutput.stories.tsx). Internal symbols updated (`GeneratedImagePreview``GeneratedOutputPreview`, `ImageCard``OutputCard`) and stale "image" copy in the placeholder swapped to "output". Kept the standalone-mock approach (option (a)) — the file's leading comment explicitly calls out "without Next.js dependencies", and rendering the real `GeneratedOutput` would require mocking workflow data, intersection observer, tRPC mutations, tour context, and the generated-item store.
---
## Out of scope (already done, noted here for closure)
- `BlobData.status` getter — **removed**; callers migrated to `available` / `step.status`.
- `imageMeta``outputMeta` on `BlobData`**renamed**.
- "images" → "output(s)" across class getters and doc comments — **done** on the client API surface.
- Old wrapper files (`GeneratedImage.tsx`, `GeneratedImageLightbox.tsx`, `Blob*.tsx`) — **deleted**; consumers updated.
- Component renames (`Blob*``GeneratedOutput*`) — **done**.
- `workflowId` / `stepName` / `jobId` on normalized wire type — **dropped** (derived via parent refs or unused).
- `previewUrl` / `previewUrlExpiresAt`**moved** to `NormalizedImageOutput` / `ImageBlob` only.
- Legacy `metadata.images``metadata.output`**dual-key with client-side merge** (see item 4); writes go to `output`, reads merge both.
- Optimistic-update tag-sync regression on legacy workflows — **fixed**: tag-sync now operates on the merged post-patch state.
- Filter-cache pruning in `updateImages`**added**: workflows that no longer match a cache's filter tags are dropped from that cache.
- `GenerationProvider` queue tracking across filter changes — **fixed**: now uses `ignoreFilters: true` so queue/canGenerate/hasGeneratedImages reflect all in-flight workflows.
- `BlobData.errored` + UI error card for `step.status === 'succeeded' && !available`**added** (still pending root-cause investigation; see item 7).
@@ -660,7 +660,7 @@ function GeneratorTab({ challenge }: { challenge?: ChallengeDetail }) {
useGetTextToImageRequests({ tags: [WORKFLOW_TAGS.IMAGE] }, { enabled: !!currentUser, ignoreFilters: true });
const generatedMedia = useMemo(
() => data.flatMap((wf) => wf.succeededImages.filter((x) => x.available)),
() => data.flatMap((wf) => wf.succeededOutput.filter((x) => x.available)),
[data]
);
@@ -1,13 +1,13 @@
import dynamic from 'next/dynamic';
import { routedDialogDictionary } from '~/components/Dialog/routed-dialog/utils';
const GeneratedImageLightbox = dynamic(
() => import('~/components/ImageGeneration/GeneratedImageLightbox'),
const GeneratedOutputLightbox = dynamic(
() => import('~/components/ImageGeneration/GeneratedOutputLightbox'),
{ ssr: false }
);
const generatedImageDialog = routedDialogDictionary.addItem('generatedImage', {
component: GeneratedImageLightbox,
component: GeneratedOutputLightbox,
resolve: (query, { imageId, workflowId }) => ({
query: { ...query, imageId, workflowId },
}),
@@ -1627,7 +1627,7 @@ SourceImageUploadMultiple.Image = function ImagePreview({
) : (
// Desktop: Full hover overlay
<div
className="absolute inset-0 flex cursor-pointer items-center justify-center bg-black/50 opacity-0 transition-opacity group-hover:opacity-100"
className="absolute inset-0 z-20 flex cursor-pointer items-center justify-center bg-black/50 opacity-0 transition-opacity group-hover:opacity-100"
onClick={handleOpenDrawingEditor}
>
<div className="flex items-center gap-2 rounded-md bg-white/90 px-3 py-2 text-dark-9">
+3 -3
View File
@@ -1,7 +1,7 @@
import { Alert, Center, Loader, Stack, Text, Anchor } from '@mantine/core';
import { IconInbox } from '@tabler/icons-react';
import { useMemo } from 'react';
import { GeneratedImage } from '~/components/ImageGeneration/GeneratedImage';
import { GeneratedOutput } from '~/components/ImageGeneration/GeneratedOutput';
import {
matchesMarkerTags,
useGetTextToImageRequestsImages,
@@ -21,7 +21,7 @@ export function Feed() {
const images = useMemo(
() =>
requests.flatMap((r) =>
r.succeededImages.filter((img) => matchesMarkerTags(img, markerTags))
r.succeededOutput.filter((img) => matchesMarkerTags(img, markerTags))
),
[requests, markerTags]
);
@@ -84,7 +84,7 @@ export function Feed() {
{/* <GeneratedImagesBuzzPrompt /> */}
<div className={classes.grid} data-testid="generation-feed-list">
{images.map((image) => (
<GeneratedImage key={`${image.workflow.id}_${image.id}`} image={image} />
<GeneratedOutput key={`${image.workflow.id}_${image.id}`} image={image} />
))}
</div>
@@ -0,0 +1,13 @@
import { IconMusic } from '@tabler/icons-react';
import type { AudioBlob } from '~/shared/orchestrator/workflow-data';
export function GeneratedAudioOutput({ image }: { image: AudioBlob }) {
return (
<div className="flex size-full flex-col items-center justify-center gap-4 bg-dark-6 p-4">
<IconMusic size={48} className="text-dimmed" />
{/* eslint-disable-next-line jsx-a11y/media-has-caption */}
<audio src={image.url} controls className="w-full max-w-[280px]" preload="metadata" />
</div>
);
}
@@ -1,480 +0,0 @@
import { Checkbox, Menu } from '@mantine/core';
import {
IconDotsVertical,
IconHeart,
IconInfoCircle,
IconThumbDown,
IconThumbUp,
IconWand,
} from '@tabler/icons-react';
import clsx from 'clsx';
import type { DragEvent, MouseEvent } from 'react';
import { useState } from 'react';
import { dialogStore } from '~/components/Dialog/dialogStore';
import { EdgeMedia2 } from '~/components/EdgeMedia/EdgeMedia';
import { useGeneratedItemStore } from '~/components/Generation/stores/generated-item.store';
import { orchestratorImageSelect } from '~/components/ImageGeneration/utils/generationImage.select';
import { useUpdateImageStepMetadata } from '~/components/ImageGeneration/utils/generationRequestHooks';
import { useInViewDynamic } from '~/components/IntersectionObserver/IntersectionObserverProvider';
import { LegacyActionIcon } from '~/components/LegacyActionIcon/LegacyActionIcon';
import { ImageMetaPopover } from '~/components/ImageMeta/ImageMeta';
import { imageGenerationDrawerZIndex } from '~/shared/constants/app-layout.constants';
import { TextToImageQualityFeedbackModal } from '~/components/Modals/GenerationQualityFeedbackModal';
import { useTourContext } from '~/components/Tours/ToursProvider';
import { TwCard } from '~/components/TwCard/TwCard';
import { GeneratedItemWorkflowMenu } from '~/components/generation_v2/GeneratedItemWorkflowMenu';
import type { BlobData } from '~/shared/orchestrator/workflow-data';
import { mediaDropzoneData } from '~/store/post-image-transmitter.store';
import { getStepMeta } from './GenerationForm/generation.utils';
import classes from './GeneratedImage.module.css';
import GeneratedImageLightbox from '~/components/ImageGeneration/GeneratedImageLightbox';
// const GeneratedImageLightbox = dynamic(
// () => import('~/components/ImageGeneration/GeneratedImageLightbox'),
// { ssr: false }
// );
export type GeneratedImageProps = {
image: BlobData;
};
export function GeneratedImage({
image,
isLightbox,
isActiveSlide,
}: {
image: BlobData;
isLightbox?: boolean;
isActiveSlide?: boolean;
}) {
const step = image.step;
const request = image.workflow;
const [ref, inView] = useInViewDynamic({ id: image.id });
const selected = orchestratorImageSelect.useIsSelected(image);
const isSelecting = orchestratorImageSelect.useIsSelecting();
const { updateImages } = useUpdateImageStepMetadata();
const { running, helpers } = useTourContext();
const available = image.status === 'succeeded';
const toggleSelect = (checked?: boolean) => orchestratorImageSelect.toggle(image, checked);
const handleImageClick = () => {
if (!image || !available || isLightbox) return;
if (isSelecting) {
handleToggleSelect();
} else {
dialogStore.trigger({
id: 'generated-image',
component: GeneratedImageLightbox,
props: { imageId: image.id, workflowId: request.id },
});
}
};
const [state, setState] = useGeneratedItemStore({
id: `${request.id}_${step.name}_${image.id}`,
favorite: step.metadata?.images?.[image.id]?.favorite === true,
feedback: step.metadata?.images?.[image.id]?.feedback,
});
function handleToggleFeedback(newFeedback: 'liked' | 'disliked') {
const previousState = state;
setState((state) => ({
feedback: state.feedback === newFeedback ? undefined : newFeedback,
}));
const onError = () => setState(previousState);
if (state.feedback !== 'disliked' && newFeedback === 'disliked') {
dialogStore.trigger({
component: TextToImageQualityFeedbackModal,
props: {
workflowId: request.id,
imageId: image.id,
comments: step.metadata?.images?.[image.id]?.comments,
stepName: step.name,
},
});
}
updateImages(
[
{
workflowId: request.id,
stepName: step.name,
images: {
[image.id]: {
feedback: newFeedback,
},
},
},
],
onError
);
}
function handleToggleFavorite(newValue: true | false) {
const previousState = state;
setState({
favorite: newValue,
});
const onError = () => setState(previousState);
updateImages(
[
{
workflowId: request.id,
stepName: step.name,
images: {
[image.id]: {
favorite: newValue,
},
},
},
],
onError
);
}
if (image.status !== 'succeeded') return <></>;
function handleDataTransfer(e: DragEvent<HTMLVideoElement> | DragEvent<HTMLImageElement>) {
// Always use full quality URL for drag and drop, not the preview
const url = image.url;
const meta = getStepMeta(step);
if (meta) mediaDropzoneData.setData(url, meta);
e.dataTransfer.setData('text/uri-list', url);
}
function handleDragVideo(e: DragEvent<HTMLVideoElement>) {
handleDataTransfer(e);
}
function handleDragImage(e: DragEvent<HTMLImageElement>) {
handleDataTransfer(e);
}
function handleContextMenu(e: MouseEvent<HTMLImageElement | HTMLVideoElement>) {
// Swap to full quality URL before context menu shows
// so "Save Image As" saves the full quality version
const element = e.currentTarget;
const previewUrl = image.previewUrl ?? image.url;
if (image.previewUrl && 'src' in element && !isLightbox) {
element.src = image.url;
// Restore preview after context menu closes
const restore = () => {
element.src = previewUrl;
document.removeEventListener('click', restore);
document.removeEventListener('keydown', restore);
};
// Delay listener attachment to allow context menu to process
setTimeout(() => {
document.addEventListener('click', restore, { once: true });
document.addEventListener('keydown', restore, { once: true });
}, 0);
}
}
function handleToggleSelect(value = !selected) {
toggleSelect(value);
if (running && value) helpers?.next();
}
return (
<TwCard
ref={ref}
className={clsx(
'max-w-full border',
isLightbox ? 'max-h-[calc(100vh-32px)] items-center justify-center' : 'w-full self-start',
classes.imageWrapper,
selected && 'ring-2 ring-blue-5/60'
)}
style={isLightbox ? { aspectRatio: image.aspect } : undefined}
>
{!isLightbox && !inView && <div style={{ aspectRatio: image.aspect }} />}
{(isLightbox || inView) && (
<>
<div
className={clsx(
'relative flex items-center justify-center',
isLightbox ? 'max-h-[calc(100vh-32px)]' : 'max-h-full'
)}
style={{ aspectRatio: image.aspect }}
>
{
<EdgeMedia2
// Use previewUrl for rendering in queue (smaller/faster), but full url for lightbox
src={isLightbox ? image.url : image.previewUrl ?? image.url}
type={image.type}
alt=""
className={clsx('max-h-full min-h-0 w-auto max-w-full', {
['cursor-pointer']: !isLightbox,
})}
onClick={handleImageClick}
onMouseDown={(e) => {
// Always use full url when opening in new tab
if (e.button === 1) return handleAuxClick(image.url);
}}
wrapperProps={{
onClick: handleImageClick,
onMouseDown: (e) => {
// Always use full url when opening in new tab
if (e.button === 1) return handleAuxClick(image.url);
},
}}
muted={!isLightbox || !isActiveSlide}
controls={isLightbox && isActiveSlide}
disableWebm
disablePoster
imageProps={{
onDragStart: handleDragImage,
onContextMenu: handleContextMenu,
...(isLightbox && {
style: {
width: 'auto',
maxHeight: 'calc(100vh - 32px)',
maxWidth: 'calc(100vw - 32px)',
},
}),
}}
videoProps={{
onDragStart: handleDragVideo,
onContextMenu: handleContextMenu,
draggable: true,
autoPlay: true,
}}
/>
}
<div className="pointer-events-none absolute size-full shadow-[inset_0_0_2px_1px_rgba(255,255,255,0.2)]" />
{!isLightbox && !image.blockedReason && (
<label className="absolute left-3 top-3" data-tour="gen:select">
<Checkbox
className={classes.checkbox}
checked={selected}
onChange={(e) => handleToggleSelect(e.target.checked)}
/>
</label>
)}
{!image.blockedReason && (
<Menu zIndex={400} withinPortal>
<Menu.Target>
<div className="absolute right-3 top-3">
<LegacyActionIcon variant="transparent">
<IconDotsVertical
size={26}
color="#fff"
filter="drop-shadow(1px 1px 2px rgb(0 0 0 / 50%)) drop-shadow(0px 5px 15px rgb(0 0 0 / 60%))"
/>
</LegacyActionIcon>
</div>
</Menu.Target>
<Menu.Dropdown className={classes.scrollableDropdown}>
<GeneratedItemWorkflowMenu image={image} isLightbox={isLightbox} />
</Menu.Dropdown>
</Menu>
)}
{!image.blockedReason && (
<GeneratedImageActions
image={image}
state={state}
isLightbox={isLightbox}
isOverlay={!isLightbox}
onToggleFavorite={handleToggleFavorite}
onToggleFeedback={handleToggleFeedback}
/>
)}
{!isLightbox && (
<div className="absolute bottom-2 right-2">
<ImageMetaPopover
meta={step.params as any}
zIndex={imageGenerationDrawerZIndex + 1}
hideSoftware
>
<LegacyActionIcon variant="transparent" size="md">
<IconInfoCircle
color="white"
filter="drop-shadow(1px 1px 2px rgb(0 0 0 / 50%)) drop-shadow(0px 5px 15px rgb(0 0 0 / 60%))"
opacity={0.8}
strokeWidth={2.5}
size={26}
/>
</LegacyActionIcon>
</ImageMetaPopover>
</div>
)}
</div>
{!isLightbox && !image.blockedReason && (
<GeneratedImageActions
image={image}
state={state}
isMobileFooter
onToggleFavorite={handleToggleFavorite}
onToggleFeedback={handleToggleFeedback}
/>
)}
</>
)}
</TwCard>
);
}
function GeneratedImageActions({
image,
state,
isLightbox,
isOverlay,
isMobileFooter,
onToggleFavorite,
onToggleFeedback,
}: {
image: BlobData;
state: { favorite?: boolean; feedback?: string };
isLightbox?: boolean;
isOverlay?: boolean;
isMobileFooter?: boolean;
onToggleFavorite: (value: boolean) => void;
onToggleFeedback: (feedback: 'liked' | 'disliked') => void;
}) {
const [menuOpen, setMenuOpen] = useState(false);
if (isLightbox || isOverlay) {
return (
<div
className={clsx(
classes.actionsWrapper,
(menuOpen || isLightbox) && classes.actionsVisible,
isOverlay && classes.desktopOnly,
image.type === 'video' ? 'bottom-2 left-12' : 'bottom-1 left-1',
'absolute flex flex-wrap items-center gap-1 p-1'
)}
>
<LegacyActionIcon
size="md"
className={state.favorite ? classes.favoriteButton : undefined}
variant={state.favorite ? 'light' : 'subtle'}
color={state.favorite ? 'red' : 'gray'}
onClick={() => onToggleFavorite(!state.favorite)}
>
<IconHeart size={16} />
</LegacyActionIcon>
<LegacyActionIcon
size="md"
variant={state.feedback === 'liked' ? 'light' : 'subtle'}
color={state.feedback === 'liked' ? 'green' : 'gray'}
onClick={() => onToggleFeedback('liked')}
>
<IconThumbUp size={16} />
</LegacyActionIcon>
<LegacyActionIcon
size="md"
variant={state.feedback === 'disliked' ? 'light' : 'subtle'}
color={state.feedback === 'disliked' ? 'red' : 'gray'}
onClick={() => onToggleFeedback('disliked')}
>
<IconThumbDown size={16} />
</LegacyActionIcon>
<Menu
zIndex={400}
trigger="click-hover"
openDelay={100}
closeDelay={100}
transitionProps={{
transition: 'fade',
duration: 150,
}}
withinPortal
position="top"
onChange={setMenuOpen}
withArrow
>
<Menu.Target>
<LegacyActionIcon size="md">
<IconWand size={16} />
</LegacyActionIcon>
</Menu.Target>
<Menu.Dropdown className={clsx(classes.improveMenu, classes.scrollableDropdown)}>
<GeneratedItemWorkflowMenu image={image} workflowsOnly isLightbox={isLightbox} />
</Menu.Dropdown>
</Menu>
</div>
);
}
return (
<div
className={clsx(classes.actionsFooter, isMobileFooter && classes.mobileOnly, 'flex w-full')}
>
<LegacyActionIcon
className={classes.footerActionIcon}
variant={state.favorite ? 'light' : 'subtle'}
color={state.favorite ? 'red' : 'gray'}
onClick={() => onToggleFavorite(!state.favorite)}
>
<IconHeart size={16} />
</LegacyActionIcon>
<div className={classes.footerDivider} />
<LegacyActionIcon
className={classes.footerActionIcon}
variant={state.feedback === 'liked' ? 'light' : 'subtle'}
color={state.feedback === 'liked' ? 'green' : 'gray'}
onClick={() => onToggleFeedback('liked')}
>
<IconThumbUp size={16} />
</LegacyActionIcon>
<div className={classes.footerDivider} />
<LegacyActionIcon
className={classes.footerActionIcon}
variant={state.feedback === 'disliked' ? 'light' : 'subtle'}
color={state.feedback === 'disliked' ? 'red' : 'gray'}
onClick={() => onToggleFeedback('disliked')}
>
<IconThumbDown size={16} />
</LegacyActionIcon>
<div className={classes.footerDivider} />
<Menu
zIndex={400}
trigger="click"
transitionProps={{ transition: 'fade', duration: 150 }}
withinPortal
position="top"
onChange={setMenuOpen}
withArrow
>
<Menu.Target>
<LegacyActionIcon className={classes.footerActionIcon}>
<IconWand size={16} />
</LegacyActionIcon>
</Menu.Target>
<Menu.Dropdown className={clsx(classes.improveMenu, classes.scrollableDropdown)}>
<GeneratedItemWorkflowMenu image={image} workflowsOnly />
</Menu.Dropdown>
</Menu>
</div>
);
}
function handleAuxClick(url: string) {
window.open(url, '_blank');
}
@@ -42,7 +42,7 @@ export function GeneratedImageActions({
const router = useRouter();
const { data } = useGetTextToImageRequests();
const { running, helpers, returnUrl } = useTourContext();
const selectableImages = useMemo(() => data.flatMap((wf) => wf.succeededImages), [data]);
const selectableImages = useMemo(() => data.flatMap((wf) => wf.succeededOutput), [data]);
const selected = orchestratorImageSelect.useSelection();
const deselect = () => orchestratorImageSelect.setSelected([]);
const [zipping, setZipping] = useState(false);
@@ -0,0 +1,76 @@
import type { DragEvent, MouseEvent } from 'react';
import { EdgeMedia2 } from '~/components/EdgeMedia/EdgeMedia';
import type { ImageBlob, StepData } from '~/shared/orchestrator/workflow-data';
import { mediaDropzoneData } from '~/store/post-image-transmitter.store';
import { getStepMeta } from './GenerationForm/generation.utils';
export function GeneratedImageOutput({
image,
step,
isLightbox,
onClick,
}: {
image: ImageBlob;
step: StepData;
isLightbox?: boolean;
onClick?: () => void;
}) {
function handleDragImage(e: DragEvent<HTMLImageElement>) {
const url = image.url;
const meta = getStepMeta(step);
if (meta) mediaDropzoneData.setData(url, meta);
e.dataTransfer.setData('text/uri-list', url);
}
function handleContextMenu(e: MouseEvent<HTMLImageElement>) {
const element = e.currentTarget;
const previewUrl = image.previewUrl ?? image.url;
if (image.previewUrl && 'src' in element && !isLightbox) {
element.src = image.url;
const restore = () => {
element.src = previewUrl;
document.removeEventListener('click', restore);
document.removeEventListener('keydown', restore);
};
setTimeout(() => {
document.addEventListener('click', restore, { once: true });
document.addEventListener('keydown', restore, { once: true });
}, 0);
}
}
return (
<EdgeMedia2
src={isLightbox ? image.url : image.previewUrl ?? image.url}
type="image"
alt=""
className={`max-h-full min-h-0 w-auto max-w-full${!isLightbox ? ' cursor-pointer' : ''}`}
onClick={onClick}
onMouseDown={(e) => {
if (e.button === 1) window.open(image.url, '_blank');
}}
wrapperProps={{
onClick,
onMouseDown: (e) => {
if (e.button === 1) window.open(image.url, '_blank');
},
}}
imageProps={{
onDragStart: handleDragImage,
onContextMenu: handleContextMenu,
...(isLightbox && {
style: {
width: 'auto',
maxHeight: 'calc(100vh - 32px)',
maxWidth: 'calc(100vw - 32px)',
},
}),
}}
/>
);
}
@@ -12,7 +12,7 @@ import { useState } from 'react';
import classes from './GeneratedImage.module.css';
// Standalone card without Next.js dependencies
function ImageCard({ children, className }: { children: React.ReactNode; className?: string }) {
function OutputCard({ children, className }: { children: React.ReactNode; className?: string }) {
return (
<div
className={clsx(
@@ -26,7 +26,7 @@ function ImageCard({ children, className }: { children: React.ReactNode; classNa
);
}
function GeneratedImagePreview({
function GeneratedOutputPreview({
aspect = '1 / 1',
favorite = false,
feedback,
@@ -39,8 +39,8 @@ function GeneratedImagePreview({
const [fb, setFb] = useState<'liked' | 'disliked' | undefined>(feedback);
return (
<ImageCard className={classes.imageWrapper}>
{/* Image area */}
<OutputCard className={classes.imageWrapper}>
{/* Output area */}
<div className="relative flex items-center justify-center" style={{ aspectRatio: aspect }}>
{/* Placeholder gradient */}
<div
@@ -51,7 +51,7 @@ function GeneratedImagePreview({
}}
/>
<span className="relative text-xs" style={{ color: '#555' }}>
image
output
</span>
{/* Top-left checkbox */}
@@ -105,44 +105,44 @@ function GeneratedImagePreview({
<IconThumbDown size={16} />
</button>
</div>
</ImageCard>
</OutputCard>
);
}
export const Square = () => (
<div style={{ width: 240 }}>
<GeneratedImagePreview aspect="1 / 1" />
<GeneratedOutputPreview aspect="1 / 1" />
</div>
);
export const Portrait = () => (
<div style={{ width: 200 }}>
<GeneratedImagePreview aspect="2 / 3" />
<GeneratedOutputPreview aspect="2 / 3" />
</div>
);
export const Landscape = () => (
<div style={{ width: 320 }}>
<GeneratedImagePreview aspect="16 / 9" />
<GeneratedOutputPreview aspect="16 / 9" />
</div>
);
export const WithFavorite = () => (
<div style={{ width: 240 }}>
<GeneratedImagePreview aspect="1 / 1" favorite />
<GeneratedOutputPreview aspect="1 / 1" favorite />
</div>
);
export const WithFeedback = () => (
<div style={{ width: 240 }}>
<GeneratedImagePreview aspect="1 / 1" feedback="liked" />
<GeneratedOutputPreview aspect="1 / 1" feedback="liked" />
</div>
);
export const Grid = () => (
<div style={{ display: 'grid', gridTemplateColumns: 'repeat(3, 180px)', gap: 8 }}>
<GeneratedImagePreview aspect="1 / 1" />
<GeneratedImagePreview aspect="2 / 3" />
<GeneratedImagePreview aspect="1 / 1" favorite />
<GeneratedOutputPreview aspect="1 / 1" />
<GeneratedOutputPreview aspect="2 / 3" />
<GeneratedOutputPreview aspect="1 / 1" favorite />
</div>
);
@@ -0,0 +1,48 @@
import type { AudioBlob, ImageBlob, VideoBlob } from '~/shared/orchestrator/workflow-data';
import { GeneratedAudioOutput } from './GeneratedAudioOutput';
import { GeneratedImageOutput } from './GeneratedImageOutput';
import { GeneratedOutputWrapper } from './GeneratedOutputWrapper';
import { GeneratedVideoOutput } from './GeneratedVideoOutput';
export function GeneratedOutput({
image,
isLightbox,
isActiveSlide,
}: {
image: ImageBlob | VideoBlob | AudioBlob;
isLightbox?: boolean;
isActiveSlide?: boolean;
}) {
const step = image.step;
return (
<GeneratedOutputWrapper image={image} isLightbox={isLightbox} isActiveSlide={isActiveSlide}>
{({ onClick }) => {
switch (image.type) {
case 'audio':
return <GeneratedAudioOutput image={image} />;
case 'video':
return (
<GeneratedVideoOutput
image={image}
step={step}
isLightbox={isLightbox}
isActiveSlide={isActiveSlide}
onClick={onClick}
/>
);
default:
return (
<GeneratedImageOutput
image={image}
step={step}
isLightbox={isLightbox}
onClick={onClick}
/>
);
}
}}
</GeneratedOutputWrapper>
);
}
@@ -0,0 +1,156 @@
import { Menu } from '@mantine/core';
import { IconHeart, IconThumbDown, IconThumbUp, IconWand } from '@tabler/icons-react';
import clsx from 'clsx';
import { useState } from 'react';
import { LegacyActionIcon } from '~/components/LegacyActionIcon/LegacyActionIcon';
import { GeneratedItemWorkflowMenu } from '~/components/generation_v2/GeneratedItemWorkflowMenu';
import type { BlobData } from '~/shared/orchestrator/workflow-data';
import classes from './GeneratedImage.module.css';
export function GeneratedOutputActions({
output,
state,
isLightbox,
isOverlay,
isMobileFooter,
onToggleFavorite,
onToggleFeedback,
}: {
output: BlobData;
state: { favorite?: boolean; feedback?: string };
isLightbox?: boolean;
isOverlay?: boolean;
isMobileFooter?: boolean;
onToggleFavorite: (value: boolean) => void;
onToggleFeedback: (feedback: 'liked' | 'disliked') => void;
}) {
const [menuOpen, setMenuOpen] = useState(false);
if (isLightbox || isOverlay) {
return (
<div
className={clsx(
classes.actionsWrapper,
(menuOpen || isLightbox) && classes.actionsVisible,
isOverlay && classes.desktopOnly,
output.type === 'video' || output.type === 'audio'
? 'bottom-2 left-12'
: 'bottom-1 left-1',
'absolute flex flex-wrap items-center gap-1 p-1'
)}
>
<LegacyActionIcon
size="md"
className={state.favorite ? classes.favoriteButton : undefined}
variant={state.favorite ? 'light' : 'subtle'}
color={state.favorite ? 'red' : 'gray'}
onClick={() => onToggleFavorite(!state.favorite)}
>
<IconHeart size={16} />
</LegacyActionIcon>
<LegacyActionIcon
size="md"
variant={state.feedback === 'liked' ? 'light' : 'subtle'}
color={state.feedback === 'liked' ? 'green' : 'gray'}
onClick={() => onToggleFeedback('liked')}
>
<IconThumbUp size={16} />
</LegacyActionIcon>
<LegacyActionIcon
size="md"
variant={state.feedback === 'disliked' ? 'light' : 'subtle'}
color={state.feedback === 'disliked' ? 'red' : 'gray'}
onClick={() => onToggleFeedback('disliked')}
>
<IconThumbDown size={16} />
</LegacyActionIcon>
<Menu
zIndex={400}
trigger="click-hover"
openDelay={100}
closeDelay={100}
transitionProps={{
transition: 'fade',
duration: 150,
}}
withinPortal
position="top"
onChange={setMenuOpen}
withArrow
>
<Menu.Target>
<LegacyActionIcon size="md">
<IconWand size={16} />
</LegacyActionIcon>
</Menu.Target>
<Menu.Dropdown className={clsx(classes.improveMenu, classes.scrollableDropdown)}>
<GeneratedItemWorkflowMenu image={output} workflowsOnly isLightbox={isLightbox} />
</Menu.Dropdown>
</Menu>
</div>
);
}
return (
<div
className={clsx(classes.actionsFooter, isMobileFooter && classes.mobileOnly, 'flex w-full')}
>
<LegacyActionIcon
className={classes.footerActionIcon}
variant={state.favorite ? 'light' : 'subtle'}
color={state.favorite ? 'red' : 'gray'}
onClick={() => onToggleFavorite(!state.favorite)}
>
<IconHeart size={16} />
</LegacyActionIcon>
<div className={classes.footerDivider} />
<LegacyActionIcon
className={classes.footerActionIcon}
variant={state.feedback === 'liked' ? 'light' : 'subtle'}
color={state.feedback === 'liked' ? 'green' : 'gray'}
onClick={() => onToggleFeedback('liked')}
>
<IconThumbUp size={16} />
</LegacyActionIcon>
<div className={classes.footerDivider} />
<LegacyActionIcon
className={classes.footerActionIcon}
variant={state.feedback === 'disliked' ? 'light' : 'subtle'}
color={state.feedback === 'disliked' ? 'red' : 'gray'}
onClick={() => onToggleFeedback('disliked')}
>
<IconThumbDown size={16} />
</LegacyActionIcon>
<div className={classes.footerDivider} />
<Menu
zIndex={400}
trigger="click"
transitionProps={{ transition: 'fade', duration: 150 }}
withinPortal
position="top"
onChange={setMenuOpen}
withArrow
>
<Menu.Target>
<LegacyActionIcon className={classes.footerActionIcon}>
<IconWand size={16} />
</LegacyActionIcon>
</Menu.Target>
<Menu.Dropdown className={clsx(classes.improveMenu, classes.scrollableDropdown)}>
<GeneratedItemWorkflowMenu image={output} workflowsOnly />
</Menu.Dropdown>
</Menu>
</div>
);
}
@@ -12,9 +12,9 @@ import {
} from '~/components/ImageGeneration/utils/generationRequestHooks';
import { IntersectionObserverProvider } from '~/components/IntersectionObserver/IntersectionObserverProvider';
import { GeneratedImage } from './GeneratedImage';
import { GeneratedOutput } from './GeneratedOutput';
export default function GeneratedImageLightbox({
export default function GeneratedOutputLightbox({
imageId,
workflowId,
}: {
@@ -33,18 +33,14 @@ export default function GeneratedImageLightbox({
['ArrowRight', () => embla?.scrollNext()],
]);
// Build flat image list across all loaded workflows
const images = useMemo(
() =>
(requests ?? []).flatMap((r) =>
r.succeededImages.filter((img) => matchesMarkerTags(img, markerTags))
r.succeededOutput.filter((img) => matchesMarkerTags(img, markerTags))
),
[requests, markerTags]
);
// Close only when there are no images left to display across all workflows.
// Guard on `requests !== undefined` to avoid closing during the initial load
// before any data has arrived.
useEffect(() => {
if (!isLoading && requests !== undefined && images.length === 0) dialog.onClose();
}, [isLoading, requests, images.length]); // eslint-disable-line react-hooks/exhaustive-deps
@@ -57,8 +53,6 @@ export default function GeneratedImageLightbox({
);
const [slide, setSlide] = useState(initialSlide > -1 ? initialSlide : 0);
// Keep a ref so stale closures (EmblaCarouselProvider captures onSlideChange once at mount
// via an empty-deps useCallback) always read the current images array.
const imagesRef = useRef(images);
imagesRef.current = images;
@@ -70,17 +64,12 @@ export default function GeneratedImageLightbox({
}
};
// When images change, manually reInit Embla (since watchSlides is disabled) and
// restore position to the tracked image. If the tracked image was deleted,
// advance to the next available one instead.
useEffect(() => {
if (!embla) return;
const desiredIndex = images.findIndex((item) => imageKey(item) === currentImageKeyRef.current);
if (desiredIndex === -1) {
// Tracked image was removed (deleted) — navigate to the next available image.
// Don't close here; the hasWorkflow effect handles closing when appropriate.
if (images.length > 0) {
const nextIndex = Math.min(slide, images.length - 1);
currentImageKeyRef.current = imageKey(images[nextIndex]);
@@ -88,8 +77,6 @@ export default function GeneratedImageLightbox({
setSlide(nextIndex);
}
} else {
// New images may have been added — reInit with startIndex so Embla registers
// the new slides AND positions itself at the correct image in one step.
embla.reInit({ startIndex: desiredIndex });
if (desiredIndex !== slide) setSlide(desiredIndex);
}
@@ -136,7 +123,7 @@ export default function GeneratedImageLightbox({
{image.url &&
(Math.abs(index - slide) <= 1 ||
Math.abs(index - slide) >= images.length - 1) && (
<GeneratedImage image={image} isLightbox isActiveSlide={index === slide} />
<GeneratedOutput image={image} isLightbox isActiveSlide={index === slide} />
)}
</Embla.Slide>
))}
@@ -0,0 +1,248 @@
import { Checkbox, Menu, Text } from '@mantine/core';
import { IconAlertTriangle, IconDotsVertical, IconInfoCircle } from '@tabler/icons-react';
import clsx from 'clsx';
import type { ReactNode } from 'react';
import { dialogStore } from '~/components/Dialog/dialogStore';
import { useGeneratedItemStore } from '~/components/Generation/stores/generated-item.store';
import { orchestratorImageSelect } from '~/components/ImageGeneration/utils/generationImage.select';
import { useUpdateImageStepMetadata } from '~/components/ImageGeneration/utils/generationRequestHooks';
import { useInViewDynamic } from '~/components/IntersectionObserver/IntersectionObserverProvider';
import { LegacyActionIcon } from '~/components/LegacyActionIcon/LegacyActionIcon';
import { ImageMetaPopover } from '~/components/ImageMeta/ImageMeta';
import { imageGenerationDrawerZIndex } from '~/shared/constants/app-layout.constants';
import { TextToImageQualityFeedbackModal } from '~/components/Modals/GenerationQualityFeedbackModal';
import { useTourContext } from '~/components/Tours/ToursProvider';
import { TwCard } from '~/components/TwCard/TwCard';
import { GeneratedItemWorkflowMenu } from '~/components/generation_v2/GeneratedItemWorkflowMenu';
import type { AudioBlob, ImageBlob, VideoBlob } from '~/shared/orchestrator/workflow-data';
import { GeneratedOutputActions } from './GeneratedOutputActions';
import classes from './GeneratedImage.module.css';
export function GeneratedOutputWrapper({
image,
isLightbox,
isActiveSlide,
children,
}: {
image: ImageBlob | VideoBlob | AudioBlob;
isLightbox?: boolean;
isActiveSlide?: boolean;
children: (props: { onClick: () => void }) => ReactNode;
}) {
const step = image.step;
const request = image.workflow;
const [ref, inView] = useInViewDynamic({ id: image.id });
const selected = orchestratorImageSelect.useIsSelected(image);
const isSelecting = orchestratorImageSelect.useIsSelecting();
const { updateImages } = useUpdateImageStepMetadata();
const { running, helpers } = useTourContext();
const available = image.available;
const toggleSelect = (checked?: boolean) => orchestratorImageSelect.toggle(image, checked);
const handleClick = () => {
if (!image || !available || isLightbox) return;
if (isSelecting) {
handleToggleSelect();
} else {
// Lazy import to avoid circular dependency
import('./GeneratedOutputLightbox').then(({ default: GeneratedOutputLightbox }) => {
dialogStore.trigger({
id: 'generated-image',
component: GeneratedOutputLightbox,
props: { imageId: image.id, workflowId: request.id },
});
});
}
};
// Read via `outputMeta` so legacy `metadata.images` data is merged with the
// current `metadata.output` data. Direct reads off `step.metadata.output` would
// miss legacy workflows whose per-output state still lives under `images`.
const outputMeta = image.outputMeta;
const [state, setState] = useGeneratedItemStore({
id: `${request.id}_${step.name}_${image.id}`,
favorite: outputMeta?.favorite === true,
feedback: outputMeta?.feedback,
});
function handleToggleFeedback(newFeedback: 'liked' | 'disliked') {
const previousState = state;
setState((state) => ({
feedback: state.feedback === newFeedback ? undefined : newFeedback,
}));
const onError = () => setState(previousState);
if (state.feedback !== 'disliked' && newFeedback === 'disliked') {
dialogStore.trigger({
component: TextToImageQualityFeedbackModal,
props: {
workflowId: request.id,
imageId: image.id,
comments: outputMeta?.comments,
stepName: step.name,
},
});
}
updateImages(
[
{
workflowId: request.id,
stepName: step.name,
images: { [image.id]: { feedback: newFeedback } },
},
],
onError
);
}
function handleToggleFavorite(newValue: boolean) {
const previousState = state;
setState({ favorite: newValue });
const onError = () => setState(previousState);
updateImages(
[
{
workflowId: request.id,
stepName: step.name,
images: { [image.id]: { favorite: newValue } },
},
],
onError
);
}
function handleToggleSelect(value = !selected) {
toggleSelect(value);
if (running && value) helpers?.next();
}
const aspectRatio = image.aspect;
// Step terminated but the blob never materialized — show an error card in the slot.
if (!image.available && image.errored) {
return (
<TwCard
className="flex flex-col items-center justify-center gap-2 border border-red-5 p-3"
style={{ aspectRatio }}
>
<IconAlertTriangle size={28} className="text-red-5" />
<Text c="red" fw="bold" align="center" size="sm">
Generation failed
</Text>
<Text c="dimmed" align="center" size="xs">
The worker finished without producing this output.
</Text>
</TwCard>
);
}
// Still processing (no terminal state yet) — render nothing so the placeholder card above takes the slot.
if (!image.available) return <></>;
return (
<TwCard
ref={ref}
className={clsx(
'max-w-full border',
isLightbox ? 'max-h-[calc(100vh-32px)] items-center justify-center' : 'w-full self-start',
classes.imageWrapper,
selected && 'ring-2 ring-blue-5/60'
)}
style={isLightbox ? { aspectRatio } : undefined}
>
{!isLightbox && !inView && <div style={{ aspectRatio }} />}
{(isLightbox || inView) && (
<>
<div
className={clsx(
'relative flex items-center justify-center',
isLightbox ? 'max-h-[calc(100vh-32px)]' : 'max-h-full'
)}
style={{ aspectRatio }}
>
{children({ onClick: handleClick })}
<div className="pointer-events-none absolute size-full shadow-[inset_0_0_2px_1px_rgba(255,255,255,0.2)]" />
{!isLightbox && !image.blockedReason && (
<label className="absolute left-3 top-3" data-tour="gen:select">
<Checkbox
className={classes.checkbox}
checked={selected}
onChange={(e) => handleToggleSelect(e.target.checked)}
/>
</label>
)}
{!image.blockedReason && (
<Menu zIndex={400} withinPortal>
<Menu.Target>
<div className="absolute right-3 top-3">
<LegacyActionIcon variant="transparent">
<IconDotsVertical
size={26}
color="#fff"
filter="drop-shadow(1px 1px 2px rgb(0 0 0 / 50%)) drop-shadow(0px 5px 15px rgb(0 0 0 / 60%))"
/>
</LegacyActionIcon>
</div>
</Menu.Target>
<Menu.Dropdown className={classes.scrollableDropdown}>
<GeneratedItemWorkflowMenu image={image} isLightbox={isLightbox} />
</Menu.Dropdown>
</Menu>
)}
{!image.blockedReason && (
<GeneratedOutputActions
output={image}
state={state}
isLightbox={isLightbox}
isOverlay={!isLightbox}
onToggleFavorite={handleToggleFavorite}
onToggleFeedback={handleToggleFeedback}
/>
)}
{!isLightbox && (
<div className="absolute bottom-2 right-2">
<ImageMetaPopover
meta={step.params as any}
zIndex={imageGenerationDrawerZIndex + 1}
hideSoftware
>
<LegacyActionIcon variant="transparent" size="md">
<IconInfoCircle
color="white"
filter="drop-shadow(1px 1px 2px rgb(0 0 0 / 50%)) drop-shadow(0px 5px 15px rgb(0 0 0 / 60%))"
opacity={0.8}
strokeWidth={2.5}
size={26}
/>
</LegacyActionIcon>
</ImageMetaPopover>
</div>
)}
</div>
{!isLightbox && !image.blockedReason && (
<GeneratedOutputActions
output={image}
state={state}
isMobileFooter
onToggleFavorite={handleToggleFavorite}
onToggleFeedback={handleToggleFeedback}
/>
)}
</>
)}
</TwCard>
);
}
@@ -0,0 +1,56 @@
import type { DragEvent } from 'react';
import { EdgeMedia2 } from '~/components/EdgeMedia/EdgeMedia';
import type { StepData, VideoBlob } from '~/shared/orchestrator/workflow-data';
import { mediaDropzoneData } from '~/store/post-image-transmitter.store';
import { getStepMeta } from './GenerationForm/generation.utils';
export function GeneratedVideoOutput({
image,
step,
isLightbox,
isActiveSlide,
onClick,
}: {
image: VideoBlob;
step: StepData;
isLightbox?: boolean;
isActiveSlide?: boolean;
onClick?: () => void;
}) {
function handleDragVideo(e: DragEvent<HTMLVideoElement>) {
const url = image.url;
const meta = getStepMeta(step);
if (meta) mediaDropzoneData.setData(url, meta);
e.dataTransfer.setData('text/uri-list', url);
}
return (
<EdgeMedia2
src={image.url}
type="video"
alt=""
className={`max-h-full min-h-0 w-auto max-w-full${!isLightbox ? ' cursor-pointer' : ''}`}
onClick={onClick}
onMouseDown={(e) => {
if (e.button === 1) window.open(image.url, '_blank');
}}
wrapperProps={{
onClick,
onMouseDown: (e) => {
if (e.button === 1) window.open(image.url, '_blank');
},
}}
muted={!isLightbox || !isActiveSlide}
controls={isLightbox && isActiveSlide}
disableWebm
disablePoster
videoProps={{
onDragStart: handleDragVideo,
draggable: true,
autoPlay: true,
}}
/>
);
}
@@ -67,9 +67,13 @@ export function useGenerationContext<T>(selector: (state: GenerationState) => T)
export function GenerationProvider({ children }: { children: React.ReactNode }) {
const storeRef = useRef<GenerationStore>();
const opened = useGenerationPanelStore((state) => state.opened);
// Bypass the user's marker / tag filter: the queue snackbar / canGenerate /
// hasGeneratedImages need to reflect ALL in-flight and completed workflows,
// not just those matching the currently-selected filter (e.g. "liked").
const { data: requests, isLoading } = useGetTextToImageRequests(undefined, {
enabled: opened,
includeTags: false,
ignoreFilters: true,
});
const generationStatus = useGenerationStatus();
@@ -163,7 +167,7 @@ export function GenerationProvider({ children }: { children: React.ReactNode })
useEffect(() => {
const store = storeRef.current;
if (!store) return;
const hasGeneratedImages = requests.some((r) => r.steps.some((s) => s.images.length > 0));
const hasGeneratedImages = requests.some((r) => r.steps.some((s) => s.output.length > 0));
store.setState({ requestsLoading: isLoading, hasGeneratedImages });
}, [requests, isLoading]);
// #endregion
+33 -31
View File
@@ -26,7 +26,7 @@ import {
import { NextLink as Link, NextLink } from '~/components/NextLink/NextLink';
import dayjs from '~/shared/utils/dayjs';
import { useEffect, useState } from 'react';
import { GeneratedImage } from '~/components/ImageGeneration/GeneratedImage';
import { GeneratedOutput } from '~/components/ImageGeneration/GeneratedOutput';
import { GenerationDetails } from '~/components/ImageGeneration/GenerationDetails';
import {
useGenerationConfig,
@@ -98,12 +98,12 @@ export function QueueItem({
const params = request.params;
const resources = request.resources;
const allImages = request.steps.flatMap((s) => s.images);
const allImages = request.steps.flatMap((s) => s.output);
const stepErrors = request.steps.flatMap((s) => s.errors ?? []);
const failureReason = stepErrors.length
? stepErrors.join(',\n')
: allImages.find((x) => x.status === 'failed' && x.blockedReason)?.blockedReason;
: allImages.find((x) => !x.available && x.blockedReason)?.blockedReason;
const processing = status === 'processing';
const pending = orchestratorPendingStatuses.includes(status);
@@ -324,28 +324,30 @@ export function QueueItem({
)}
{stepDisplay === 'separate' ? (
request.steps.map((step) => {
const stepConfig =
workflowConfigs[step.params.workflow as keyof typeof workflowConfigs];
return (
<div key={step.name} className="flex flex-col gap-2">
<Text size="xs" c="dimmed" fw={500}>
{stepConfig?.label ?? step.name}
</Text>
<StepImages
step={step}
request={request}
features={features}
pending={pending}
processing={processing}
queuePosition={queuePosition}
markerTags={markerTags}
/>
</div>
);
})
request.steps
.filter((step) => !step.suppressOutput)
.map((step) => {
const stepConfig =
workflowConfigs[step.params.workflow as keyof typeof workflowConfigs];
return (
<div key={step.name} className="flex flex-col gap-2">
<Text size="xs" c="dimmed" fw={500}>
{stepConfig?.label ?? step.name}
</Text>
<StepOutputs
step={step}
request={request}
features={features}
pending={pending}
processing={processing}
queuePosition={queuePosition}
markerTags={markerTags}
/>
</div>
);
})
) : (
<StepImages
<StepOutputs
step={null}
request={request}
features={features}
@@ -425,7 +427,7 @@ function ResourceRow({ resource }: { resource: GenerationResource }) {
* Renders the image grid for a single step or all steps (inline mode).
* When `step` is null, renders all workflow images flattened.
*/
function StepImages({
function StepOutputs({
step,
request,
features,
@@ -442,14 +444,14 @@ function StepImages({
queuePosition?: WorkflowData['steps'][number]['queuePosition'];
markerTags?: string[];
}) {
const images = step ? step.images : request.steps.flatMap((s) => s.images);
const allDisplayImages = step ? step.displayImages : request.displayImages;
const images = step ? step.output : request.steps.flatMap((s) => s.output);
const allDisplayImages = step ? step.displayOutput : request.displayOutput;
const displayImages = allDisplayImages.filter((img) => matchesMarkerTags(img, markerTags));
const blockedReasons = step ? step.blockedReasons : request.blockedReasons;
const stepFailure = step
? step.errors?.join(',\n') ||
step.images.find((x) => x.status === 'failed' && x.blockedReason)?.blockedReason
step.output.find((x) => !x.available && x.blockedReason)?.blockedReason
: undefined;
return (
@@ -461,17 +463,17 @@ function StepImages({
})}
>
{displayImages.map((image) => (
<GeneratedImage key={image.id} image={image} />
<GeneratedOutput key={image.id} image={image} />
))}
<BlockedBlocks
blockedReasons={blockedReasons}
workflowId={request.id}
transactions={request.transactions}
/>
{(pending || processing) && images[0] && (
{(pending || processing) && (
<TwCard
className="items-center justify-center border"
style={{ aspectRatio: images[0].aspect }}
style={{ aspectRatio: images[0]?.aspect ?? 1 }}
>
{processing && (
<>
@@ -12,6 +12,16 @@ import type {
GeneratedImageStepMetadata,
TextToImageStepImageMetadata,
} from '~/server/schema/orchestrator/textToImage.schema';
/**
* Client-facing step metadata shape for jsonPatch path typing: same as
* `GeneratedImageStepMetadata` but with the per-output dictionary under the
* client name (`output`) instead of the legacy `images` key. Patches written
* against this type target `output/*` paths, which the orchestrator stores as-is.
*/
type ClientStepMetadata = Omit<GeneratedImageStepMetadata, 'images'> & {
output?: GeneratedImageStepMetadata['images'];
};
import type {
PatchWorkflowParams,
PatchWorkflowStepParams,
@@ -20,8 +30,10 @@ import type {
} from '~/server/schema/orchestrator/workflows.schema';
import type { BlobData } from '~/shared/orchestrator/workflow-data';
import { WorkflowData } from '~/shared/orchestrator/workflow-data';
import type { WorkflowStepFormatted } from '~/server/services/orchestrator/common';
import type { queryGeneratedImageWorkflows2 } from '~/server/services/orchestrator/orchestration-new.service';
import type {
NormalizedStep,
queryGeneratedImageWorkflows2,
} from '~/server/services/orchestrator/orchestration-new.service';
import type {
IWorkflow,
IWorkflowsInfinite,
@@ -41,7 +53,7 @@ export type InfiniteTextToImageRequests = InfiniteData<
/** Check whether a BlobData image passes the active marker-tag filter. */
export function matchesMarkerTags(image: BlobData, tags?: string[]): boolean {
if (!tags?.length) return true;
const meta = image.imageMeta;
const meta = image.outputMeta;
if (tags.includes(WORKFLOW_TAGS.FAVORITE) && !meta?.favorite) return false;
if (tags.includes(WORKFLOW_TAGS.FEEDBACK.LIKED) && meta?.feedback !== 'liked') return false;
if (tags.includes(WORKFLOW_TAGS.FEEDBACK.DISLIKED) && meta?.feedback !== 'disliked') return false;
@@ -146,31 +158,41 @@ function updateTextToImageRequests({
cb,
input,
}: {
cb: (data: InfiniteTextToImageRequests) => void;
cb: (
data: InfiniteTextToImageRequests,
queryInput: z.input<typeof workflowQuerySchema> | undefined
) => void;
input?: z.input<typeof workflowQuerySchema>;
}) {
const queryKey = getQueryKey(trpc.orchestrator.queryGeneratedImages);
queryClient.setQueriesData(
{
queryKey,
exact: false,
predicate: (data: any) => {
if (input) {
const queryInput = data.queryKey[1]?.input ?? {};
for (const key in input) {
if (queryInput[key] !== (input as any)[key]) return false;
}
// Iterate per-cache so the callback can see each cache's filter input
// (needed when a mutation should drop a workflow from a cache whose filter it
// no longer matches — e.g. unliking while filtered to liked).
const entries = queryClient.getQueriesData<InfiniteTextToImageRequests>({
queryKey,
exact: false,
});
for (const [qk, data] of entries) {
if (!data) continue;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const queryInput = (qk as any)?.[1]?.input as z.input<typeof workflowQuerySchema> | undefined;
if (input) {
let skip = false;
for (const key in input) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
if ((queryInput as any)?.[key] !== (input as any)[key]) {
skip = true;
break;
}
return true;
},
},
(state) => {
return produce(state, (old?: InfiniteTextToImageRequests) => {
if (!old) return;
cb(old);
});
}
if (skip) continue;
}
);
const next = produce(data, (old?: InfiniteTextToImageRequests) => {
if (!old) return;
cb(old, queryInput);
});
queryClient.setQueryData(qk, next);
}
}
export function useUpdateWorkflow() {
@@ -293,57 +315,89 @@ export function useUpdateImageStepMetadata(options?: { onSuccess?: () => void })
const match = args.find((x) => x.workflowId === workflow.id);
if (!match) continue;
const { workflowId, stepName, images } = match;
for (const step of workflow.steps as WorkflowStepFormatted[]) {
for (const step of workflow.steps as unknown as NormalizedStep[]) {
if (step.name !== stepName) continue;
const metadata = step.metadata ?? {};
const jsonPatch = new JsonPatchFactory<GeneratedImageStepMetadata>();
if (!metadata.images) jsonPatch.addOperation({ op: 'add', path: 'images', value: {} });
for (const imageId in images) {
if (!metadata.images?.[imageId])
jsonPatch.addOperation({ op: 'add', path: `images/${imageId}`, value: {} });
const metadata = (step.metadata ?? {}) as Record<string, unknown>;
// Raw orchestrator keys — used to decide init ops below. `output` is the
// current canonical key; `images` is the legacy key on pre-rename workflows.
// A legacy workflow has `images` populated but no `output` — our first patch
// must create `output` (ASP.NET jsonpatch rejects `add` on a missing parent).
const rawOutput = metadata.output as
| Record<string, TextToImageStepImageMetadata>
| undefined;
const legacyImages = metadata.images as
| Record<string, TextToImageStepImageMetadata>
| undefined;
// Merged view — used for per-field toggle decisions (e.g. was this image
// already liked?) so legacy feedback values are taken into account.
const mergedOutput: Record<string, TextToImageStepImageMetadata> = {
...(legacyImages ?? {}),
...(rawOutput ?? {}),
};
const current = metadata.images?.[imageId] ?? {};
const jsonPatch = new JsonPatchFactory<ClientStepMetadata>();
// Create `output` on the orchestrator when it doesn't exist yet (legacy or brand-new workflow).
if (!rawOutput) jsonPatch.addOperation({ op: 'add', path: 'output', value: {} });
for (const imageId in images) {
// Per-id container must also be created when the `output` dict lacks this id —
// checked against RAW output, not merged (merged may be masking a legacy-only entry).
if (!rawOutput?.[imageId])
jsonPatch.addOperation({ op: 'add', path: `output/${imageId}`, value: {} });
const current = mergedOutput[imageId] ?? {};
const { hidden, feedback, comments, postId, favorite } = match.images[imageId];
if (hidden)
jsonPatch.addOperation({ op: 'add', path: `images/${imageId}/hidden`, value: true });
jsonPatch.addOperation({ op: 'add', path: `output/${imageId}/hidden`, value: true });
if (feedback) {
jsonPatch.addOperation({
op: feedback !== current.feedback ? 'add' : 'remove',
path: `images/${imageId}/feedback`,
path: `output/${imageId}/feedback`,
value: feedback,
});
}
if (comments)
jsonPatch.addOperation({
op: 'add',
path: `images/${imageId}/comments`,
path: `output/${imageId}/comments`,
value: comments,
});
if (postId)
jsonPatch.addOperation({
op: 'add',
path: `images/${imageId}/postId`,
path: `output/${imageId}/postId`,
value: postId,
});
if (favorite !== undefined) {
jsonPatch.addOperation({
op: favorite ? 'add' : 'remove',
path: `images/${imageId}/favorite`,
path: `output/${imageId}/favorite`,
value: true,
});
}
}
const clone = cloneDeep(metadata);
const clone = cloneDeep(metadata) as Record<string, unknown>;
applyPatch(clone, jsonPatch.operations);
const patchedImages = clone.images ?? {};
const patchedOutput =
(clone.output as Record<string, TextToImageStepImageMetadata> | undefined) ?? {};
// Per-id merge of legacy and patched state. Used for tag-sync checks and the
// "all hidden?" deletion check so legacy feedback/favorite/hidden flags are
// counted alongside the new writes.
const patchedLegacy =
(clone.images as Record<string, TextToImageStepImageMetadata> | undefined) ?? {};
const mergedPatched: Record<string, TextToImageStepImageMetadata> = {
...patchedLegacy,
};
for (const id in patchedOutput) {
mergedPatched[id] = { ...mergedPatched[id], ...patchedOutput[id] };
}
// first check if the workflow should be deleted
const hiddenCount = Object.values(patchedImages).filter((x) => x.hidden).length;
if (step.images.length === hiddenCount) {
const hiddenCount = Object.values(mergedPatched).filter((x) => x.hidden).length;
if (step.output.length === hiddenCount) {
toDelete.push(workflow.id);
} else {
const images = removeEmpty(patchedImages);
const images = removeEmpty(patchedOutput);
// return transformed data
updated.push({ workflowId, stepName, images });
@@ -351,9 +405,11 @@ export function useUpdateImageStepMetadata(options?: { onSuccess?: () => void })
const hasTagLike = workflow.tags.includes(WORKFLOW_TAGS.FEEDBACK.LIKED);
const hasTagDislike = workflow.tags.includes(WORKFLOW_TAGS.FEEDBACK.DISLIKED);
const hasFavoriteImages = Object.values(images).some((x) => x.favorite);
const hasLikedImages = Object.values(images).some((x) => x.feedback === 'liked');
const hasDislikedImages = Object.values(images).some((x) => x.feedback === 'disliked');
const hasFavoriteImages = Object.values(mergedPatched).some((x) => x.favorite);
const hasLikedImages = Object.values(mergedPatched).some((x) => x.feedback === 'liked');
const hasDislikedImages = Object.values(mergedPatched).some(
(x) => x.feedback === 'disliked'
);
if (hasTagFavorite && !hasFavoriteImages) {
tags.push({ workflowId, tag: WORKFLOW_TAGS.FAVORITE, op: 'remove' });
@@ -381,7 +437,8 @@ export function useUpdateImageStepMetadata(options?: { onSuccess?: () => void })
// Optimistically update the cache before mutation to ensure UI updates
// even if the component unmounts (e.g., menu closing)
updateTextToImageRequests({
cb: (old) => {
cb: (old, queryInput) => {
const filterTags = (queryInput?.tags ?? []) as string[];
for (const page of old.pages) {
page.items = page.items.filter((x) => !toDelete.includes(x.id));
for (const workflow of page.items) {
@@ -399,10 +456,18 @@ export function useUpdateImageStepMetadata(options?: { onSuccess?: () => void })
if (!toUpdate.length) continue;
for (const step of workflow.steps) {
const images = toUpdate.find((x) => x.stepName === step.name)?.images;
if (images) step.metadata = { ...step.metadata, images };
const output = toUpdate.find((x) => x.stepName === step.name)?.images;
if (output) step.metadata = { ...step.metadata, output };
}
}
// Drop workflows that no longer match this cache's filter tags
// (e.g. unliking an image causes the workflow to lose `feedback:liked`
// and we're viewing the "liked" filter — the workflow should disappear).
if (filterTags.length) {
page.items = page.items.filter((workflow) =>
filterTags.every((tag) => workflow.tags.includes(tag))
);
}
}
},
});
@@ -66,13 +66,13 @@ export async function updateWorkflowsStatus(workflowIds: string[]) {
// (e.g. Wan 2.2 interpolation) where the later step starts with zero images
// and only materializes outputs on completion — the previous per-index
// loop only updated existing entries and dropped new ones until reload.
for (const [index, image] of step.images.entries()) {
const imageMatch = stepMatch.images.find((x) => x.id === image.id);
if (imageMatch) step.images[index] = imageMatch;
for (const [index, item] of step.output.entries()) {
const itemMatch = stepMatch.output.find((x) => x.id === item.id);
if (itemMatch) step.output[index] = itemMatch;
}
const existingIds = new Set(step.images.map((x) => x.id));
for (const image of stepMatch.images) {
if (!existingIds.has(image.id)) step.images.push(image);
const existingIds = new Set(step.output.map((x) => x.id));
for (const item of stepMatch.output) {
if (!existingIds.has(item.id)) step.output.push(item);
}
}
}
@@ -1,7 +1,7 @@
import { Alert, Badge, Card, Center, Drawer, Loader, Stack, Text } from '@mantine/core';
import { IconInbox } from '@tabler/icons-react';
import { useMemo } from 'react';
import { GeneratedImage } from '~/components/ImageGeneration/GeneratedImage';
import { GeneratedOutput } from '~/components/ImageGeneration/GeneratedOutput';
import { GenerationDetails } from '~/components/ImageGeneration/GenerationDetails';
import { GenerationStatusBadge } from '~/components/ImageGeneration/GenerationStatusBadge';
import { InViewLoader } from '~/components/InView/InViewLoader';
@@ -119,14 +119,14 @@ function UserGenerationItem({ request }: { request: WorkflowData }) {
const step = request.steps[0];
const { status } = request;
const params = step.params;
const images = step.images;
const images = step.output;
const { prompt, ...details } = params as Record<string, unknown>;
const { data: workflowDefinitions } = trpc.generation.getWorkflowDefinitions.useQuery();
const workflowDefinition = workflowDefinitions?.find((x) => x.key === (params as any).workflow);
const displayImages = step.succeededImages;
const displayImages = step.succeededOutput;
const blockedCount = step.blockedCount;
return (
@@ -168,7 +168,7 @@ function UserGenerationItem({ request }: { request: WorkflowData }) {
{displayImages.length > 0 && (
<div className="grid grid-cols-3 gap-2">
{displayImages.map((image, index) => (
<GeneratedImage key={index} image={image} />
<GeneratedOutput key={index} image={image} />
))}
</div>
)}
@@ -171,7 +171,7 @@ export default function ImageSelectModal({
});
const generatedMedia = useMemo(
() => (generationData ?? []).flatMap((wf) => wf.succeededImages.filter((x) => x.available)),
() => (generationData ?? []).flatMap((wf) => wf.succeededOutput.filter((x) => x.available)),
[generationData]
);
@@ -260,7 +260,7 @@ export function ImageUploadMultipleInput({
{previewItems.map((item, i) => (
<div
key={i}
className={clsx('w-[200px]', imageLayout !== 'wrap' && 'shrink-0')}
className={clsx('relative w-[200px]', imageLayout !== 'wrap' && 'shrink-0')}
>
<SourceImageUploadMultiple.Image index={i} {...item} />
{imageAnnotations?.[i] && (
@@ -2,7 +2,11 @@ import { describe, it, expect } from 'vitest';
import { buildStepSource, resolveStepSource } from '../workflow-metadata';
import { getStepParams, getStepResources, WorkflowData, StepData } from '../index';
import type { WorkflowDataOptions } from '../index';
import type { NormalizedStep, NormalizedWorkflow } from '../orchestration-new.service';
import type {
NormalizedStep,
NormalizedWorkflow,
NormalizedWorkflowMetadata,
} from '../orchestration-new.service';
const defaultOptions: WorkflowDataOptions = {
domain: { green: false, blue: false, red: false } as any,
@@ -274,7 +278,7 @@ function makeStep(metadata: Partial<NormalizedStep['metadata']>): NormalizedStep
return {
$type: 'textToImage',
name: '$0',
images: [],
output: [],
metadata: { ...metadata },
} as NormalizedStep;
}
@@ -293,6 +297,11 @@ function makeWorkflow(
} as NormalizedWorkflow;
}
/** Build a bare WorkflowData for tests that only need metadata fallback behavior. */
function makeWorkflowData(metadata?: NormalizedWorkflowMetadata): WorkflowData {
return new WorkflowData({ steps: [], metadata } as any, defaultOptions);
}
describe('getStepParams', () => {
it('returns step params when present', () => {
const step = makeStep({ params: { prompt: 'step prompt', steps: 30 } });
@@ -505,82 +514,82 @@ describe('WorkflowData', () => {
describe('StepData', () => {
it('returns step params when present', () => {
const step = makeStep({ params: { prompt: 'step prompt' } });
const wfMeta = { params: { prompt: 'wf prompt' }, resources: [] };
const sd = new StepData(step, wfMeta);
const wf = makeWorkflowData({ params: { prompt: 'wf prompt' }, resources: [] });
const sd = new StepData(step, wf);
expect(sd.params).toEqual({ prompt: 'step prompt' });
});
it('falls back to workflow metadata when step has no params', () => {
const step = makeStep({});
const wfMeta = { params: { prompt: 'wf prompt', steps: 20 }, resources: [] };
const sd = new StepData(step, wfMeta);
const wf = makeWorkflowData({ params: { prompt: 'wf prompt', steps: 20 }, resources: [] });
const sd = new StepData(step, wf);
expect(sd.params).toEqual({ prompt: 'wf prompt', steps: 20 });
});
it('returns step resources when present', () => {
const step = makeStep({ resources: [{ id: 1 }] as any });
const wfMeta = { params: {}, resources: [{ id: 2 }] as any };
const sd = new StepData(step, wfMeta);
const wf = makeWorkflowData({ params: {}, resources: [{ id: 2 }] as any });
const sd = new StepData(step, wf);
expect(sd.resources).toEqual([{ id: 1 }]);
});
it('falls back to workflow resources when step has none', () => {
const step = makeStep({});
const wfMeta = { params: {}, resources: [{ id: 2 }] as any };
const sd = new StepData(step, wfMeta);
const wf = makeWorkflowData({ params: {}, resources: [{ id: 2 }] as any });
const sd = new StepData(step, wf);
expect(sd.resources).toEqual([{ id: 2 }]);
});
it('resolves remixOfId with fallback', () => {
const step = makeStep({});
const wfMeta = { params: {}, resources: [], remixOfId: 42 };
const sd = new StepData(step, wfMeta);
const wf = makeWorkflowData({ params: {}, resources: [], remixOfId: 42 });
const sd = new StepData(step, wf);
expect(sd.remixOfId).toBe(42);
});
it('step remixOfId takes precedence over workflow', () => {
const step = makeStep({ remixOfId: 10 });
const wfMeta = { params: {}, resources: [], remixOfId: 42 };
const sd = new StepData(step, wfMeta);
const wf = makeWorkflowData({ params: {}, resources: [], remixOfId: 42 });
const sd = new StepData(step, wf);
expect(sd.remixOfId).toBe(10);
});
it('prompt convenience accessor returns prompt from params', () => {
const step = makeStep({ params: { prompt: 'hello world' } });
const sd = new StepData(step);
const sd = new StepData(step, makeWorkflowData());
expect(sd.prompt).toBe('hello world');
});
it('prompt returns undefined when no params', () => {
const step = makeStep({});
const sd = new StepData(step);
const sd = new StepData(step, makeWorkflowData());
expect(sd.prompt).toBeUndefined();
});
it('returns defaults when constructed with undefined wfMetadata', () => {
it('returns defaults when workflow has no metadata', () => {
const step = makeStep({});
const sd = new StepData(step, undefined);
const sd = new StepData(step, makeWorkflowData());
expect(sd.params).toEqual({});
expect(sd.resources).toEqual([]);
expect(sd.remixOfId).toBeUndefined();
});
it('works with Omit<NormalizedStep, "images"> (no generic needed)', () => {
const step: Omit<NormalizedStep, 'images'> = {
it('works with Omit<NormalizedStep, "output"> (no generic needed)', () => {
const step: Omit<NormalizedStep, 'output'> = {
$type: 'textToImage',
name: '$0',
metadata: { params: { prompt: 'test' } },
} as any;
const sd = new StepData(step);
const sd = new StepData(step, makeWorkflowData());
expect(sd.params).toEqual({ prompt: 'test' });
});
@@ -1006,6 +1006,7 @@ export async function whatIfFromGraph({
// =============================================================================
import type {
AudioBlob,
ImageBlob,
NsfwLevel,
TransactionInfo,
@@ -1025,27 +1026,52 @@ import { parseAIR } from '~/shared/utils/air';
// Types
// =============================================================================
/** Normalized output (image or video) from a workflow step */
export interface NormalizedWorkflowStepOutput {
url: string;
workflowId: string;
stepName: string;
seed?: number | null;
status: WorkflowStatus;
aspect: number;
type: 'image' | 'video';
/**
* Fields shared by all normalized blob outputs.
* Note: `workflowId`, `stepName`, and `jobId` are intentionally absent on the client,
* consumers read workflow/step info via parent refs on `BlobData` (`blob.workflow.id`,
* `blob.step.name`), and `jobId` has no client-side consumers.
*/
interface NormalizedBlobBase {
id: string;
url: string;
seed?: number | null;
available: boolean;
urlExpiresAt?: string | null;
jobId?: string | null;
nsfwLevel?: NsfwLevel;
blockedReason?: string | null;
previewUrl?: string | null;
previewUrlExpiresAt?: string | null;
}
/** Normalized image output. */
export interface NormalizedImageOutput extends NormalizedBlobBase {
type: 'image';
width: number;
height: number;
aspect: number;
previewUrl?: string | null;
previewUrlExpiresAt?: string | null;
}
/** Normalized video output. */
export interface NormalizedVideoOutput extends NormalizedBlobBase {
type: 'video';
width: number;
height: number;
aspect: number;
}
/** Normalized audio output. */
export interface NormalizedAudioOutput extends NormalizedBlobBase {
type: 'audio';
duration?: number | null;
}
/** Normalized output (image, video, or audio) from a workflow step. */
export type NormalizedWorkflowStepOutput =
| NormalizedImageOutput
| NormalizedVideoOutput
| NormalizedAudioOutput;
/** Step metadata with mapped params and enriched resources */
export interface NormalizedStepMetadata {
/**
@@ -1060,7 +1086,24 @@ export interface NormalizedStepMetadata {
resources?: GenerationResource[];
/** Remix reference (legacy — new writes put this on workflow.metadata) */
remixOfId?: number;
/** Per-image metadata (favorite, feedback, hidden, etc.) */
/**
* Per-output metadata keyed by blob id (favorite, feedback, hidden, etc.).
* Current/canonical key. New writes target this.
*/
output?: Record<
string,
{
hidden?: boolean;
feedback?: 'liked' | 'disliked';
favorite?: boolean;
comments?: string;
postId?: number;
}
>;
/**
* Legacy per-output metadata key. Present on workflows created before the
* rename; merged into `output` by client readers for display. Do not write.
*/
images?: Record<
string,
{
@@ -1094,8 +1137,8 @@ export interface NormalizedStep {
queuePosition?: WorkflowStepJobQueuePosition;
/** Metadata with resolved params/resources */
metadata: NormalizedStepMetadata;
/** Output images/videos */
images: NormalizedWorkflowStepOutput[];
/** Output items (image / video / audio) */
output: NormalizedWorkflowStepOutput[];
/** Step errors */
errors?: string[];
}
@@ -1232,17 +1275,20 @@ type StepWithOutput = WorkflowStep & {
images?: ImageBlob[];
video?: VideoBlob;
blobs?: ImageBlob[];
blob?: ImageBlob;
// For aceStepAudio: blob.type is 'audio' (audio-only) or 'video' (audio + cover image).
blob?: ImageBlob | VideoBlob | AudioBlob;
errors?: string[];
externalTOSViolation?: boolean;
message?: string;
};
};
type NormalizedBlobItem = ImageBlob | VideoBlob | AudioBlob;
/**
* Normalizes step output (images/videos) to a common format
* Normalizes step output (images/videos/audio) to a common format
*/
function normalizeStepOutput(step: StepWithOutput): Array<ImageBlob | VideoBlob> {
function normalizeStepOutput(step: StepWithOutput): NormalizedBlobItem[] {
const output = step.output;
if (!output) return [];
@@ -1253,12 +1299,18 @@ function normalizeStepOutput(step: StepWithOutput): Array<ImageBlob | VideoBlob>
case 'textToImage':
return output.images?.map((img) => ({ ...img, type: 'image' as const })) ?? [];
case 'imageUpscaler':
return output.blob ? [{ ...output.blob, type: 'image' as const }] : [];
return output.blob ? [{ ...(output.blob as ImageBlob), type: 'image' as const }] : [];
case 'videoGen':
case 'videoUpscaler':
case 'videoEnhancement':
case 'videoInterpolation':
return output.video ? [{ ...output.video, type: 'video' as const }] : [];
case 'aceStepAudio':
// Cover-image mode returns VideoBlob; audio-only returns AudioBlob. Discriminate on blob.type.
if (!output.blob) return [];
if (output.blob.type === 'video')
return [{ ...(output.blob as VideoBlob), type: 'video' as const }];
return [{ ...(output.blob as AudioBlob), type: 'audio' as const }];
default:
return [];
}
@@ -1268,10 +1320,9 @@ function normalizeStepOutput(step: StepWithOutput): Array<ImageBlob | VideoBlob>
* Formats step outputs into normalized images array
*/
export function formatStepOutputs(
workflowId: string,
step: StepWithOutput,
resolvedParams?: Record<string, unknown>
): { images: NormalizedWorkflowStepOutput[]; errors: string[] } {
): { output: NormalizedWorkflowStepOutput[]; errors: string[] } {
const items = normalizeStepOutput(step);
const metadata = (step.metadata as Record<string, unknown>) ?? {};
const params = resolvedParams ?? ((metadata.params ?? {}) as Record<string, unknown>);
@@ -1280,12 +1331,35 @@ export function formatStepOutputs(
params?: Record<string, unknown>;
}>;
const images: NormalizedWorkflowStepOutput[] = items.map((item, index) => {
const job = step.jobs?.find((j) => j.id === item.jobId);
let { width, height } = item;
const output: NormalizedWorkflowStepOutput[] = items.map((item, index) => {
// Common fields for every output type.
const base = {
id: item.id,
seed: seed ? seed + index : undefined,
available: item.available,
urlExpiresAt: item.urlExpiresAt,
nsfwLevel: item.nsfwLevel,
blockedReason: item.blockedReason,
};
// Try to get dimensions from various sources
if (!width || !height) {
if (item.type === 'audio') {
return {
...base,
type: 'audio' as const,
url: item.url as string,
duration: item.duration,
} satisfies NormalizedAudioOutput;
}
// image / video: resolve width/height/aspect.
// For `aceStepAudio` steps that emit a VideoBlob (audio + cover image), force aspect=1
// because cover-image dimensions aren't meaningful as a display aspect.
const isAudioCoverVideo = step.$type === 'aceStepAudio' && item.type === 'video';
let width = item.width as number | null | undefined;
let height = item.height as number | null | undefined;
if (!isAudioCoverVideo && (!width || !height)) {
// Check transformations from last to first to find dimensions
if (transformations.length > 0) {
for (let i = transformations.length - 1; i >= 0; i--) {
@@ -1374,36 +1448,47 @@ export function formatStepOutputs(
height = 512;
}
const aspect = width / height;
const aspect = isAudioCoverVideo ? 1 : width / height;
const url = item.url && item.type === 'video' ? `${item.url}.mp4` : (item.url as string);
if (item.type === 'video') {
return {
...base,
type: 'video' as const,
url,
width,
height,
aspect,
} satisfies NormalizedVideoOutput;
}
return {
...(item as ImageBlob | VideoBlob),
url: item.url && item.type === 'video' ? `${item.url}.mp4` : (item.url as string),
workflowId,
stepName: step.name,
seed: seed ? seed + index : undefined,
status: item.available ? 'succeeded' : ((job?.status ?? 'unassigned') as WorkflowStatus),
aspect,
...base,
type: 'image' as const,
url,
width,
height,
};
aspect,
previewUrl: (item as ImageBlob).previewUrl,
previewUrlExpiresAt: (item as ImageBlob).previewUrlExpiresAt,
} satisfies NormalizedImageOutput;
});
// Collect errors
const errors: string[] = [];
const output = step.output;
if (output) {
if ('errors' in output && output.errors) errors.push(...output.errors);
const stepOutput = step.output;
if (stepOutput) {
if ('errors' in stepOutput && stepOutput.errors) errors.push(...stepOutput.errors);
if (
'externalTOSViolation' in output &&
'message' in output &&
typeof output.message === 'string'
'externalTOSViolation' in stepOutput &&
'message' in stepOutput &&
typeof stepOutput.message === 'string'
) {
errors.push(output.message);
errors.push(stepOutput.message);
}
}
return { images, errors };
return { output, errors };
}
// =============================================================================
@@ -1500,11 +1585,7 @@ function formatStep(
const paramsForDimensions = finalParams ?? (workflowMetadata?.params as Record<string, unknown>);
// Format outputs
const { images, errors } = formatStepOutputs(
workflowId,
step as StepWithOutput,
paramsForDimensions
);
const { output, errors } = formatStepOutputs(step as StepWithOutput, paramsForDimensions);
return {
$type: step.$type,
@@ -1517,12 +1598,18 @@ function formatStep(
...removeEmpty({
params: finalParams,
remixOfId,
// Pass both raw keys through. Client merges `output + images` for display
// via `BlobData.outputMeta`; client's patch builder inspects `output`
// directly to decide whether to emit init ops for legacy workflows
// (legacy = `images` populated, `output` absent — first patch must create
// the `output` parent path in orchestrator storage).
output: (metadata as any).output as NormalizedStepMetadata['output'],
images: metadata.images as NormalizedStepMetadata['images'],
suppressOutput: metadata.suppressOutput as boolean | undefined,
}),
...(resolvedResources?.length ? { resources: resolvedResources } : {}),
},
images,
output,
errors: errors.length > 0 ? errors : undefined,
};
}
@@ -1709,17 +1796,13 @@ export async function getWorkflowStatusUpdate({
const paramsForDimensions = Object.keys(stepParams).length > 0 ? stepParams : wfParams;
// Format step outputs using the shared utility
const { images, errors } = formatStepOutputs(
workflowId,
step as StepWithOutput,
paramsForDimensions
);
const { output, errors } = formatStepOutputs(step as StepWithOutput, paramsForDimensions);
return {
name: step.name,
status: step.status,
completedAt: step.completedAt,
images,
output,
errors: errors.length > 0 ? errors : undefined,
};
}),
@@ -54,9 +54,6 @@ export async function patchWorkflowSteps({
const client = createOrchestratorClient(token);
await Promise.all(
input.map(async ({ workflowId, stepName, patches }) => {
// console.dir(JSON.stringify({ body: patches, path: { stepName, workflowId } }), {
// depth: null,
// });
await patchWorkflowStep({ client, body: patches, path: { stepName, workflowId } });
})
);
+177 -98
View File
@@ -1,18 +1,16 @@
import type {
ImageBlob,
VideoBlob,
NsfwLevel,
WorkflowCost,
WorkflowStatus,
} from '@civitai/client';
import type { NsfwLevel, WorkflowCost } from '@civitai/client';
import type {
NormalizedWorkflow,
NormalizedWorkflowMetadata,
NormalizedStep,
NormalizedWorkflowStepOutput,
NormalizedImageOutput,
NormalizedVideoOutput,
NormalizedAudioOutput,
} from '~/server/services/orchestrator/orchestration-new.service';
import type { ColorDomain } from '~/shared/constants/domain.constants';
import { isPrivateMature, isMature } from '~/shared/constants/orchestrator.constants';
import { orchestratorCompletedStatuses } from '~/shared/constants/generation.constants';
// =============================================================================
// Defaults
@@ -33,6 +31,8 @@ export interface WorkflowDataOptions {
nsfwEnabled: boolean;
}
type BlobOptions = WorkflowDataOptions & { allowMatureContent?: boolean | null };
// =============================================================================
// WorkflowData
// =============================================================================
@@ -43,7 +43,7 @@ export interface WorkflowDataOptions {
*
* Handles the full initialization chain:
* - Wraps raw steps in StepData (metadata fallback)
* - Wraps raw images in BlobData (NSFW blocking)
* - Wraps raw outputs in BlobData subclasses (NSFW blocking)
* - Wires parent references (StepData.workflow, BlobData.step)
*/
export interface WorkflowData extends NormalizedWorkflow {
@@ -57,14 +57,14 @@ export class WorkflowData {
Object.assign(this, workflow);
// Initialize chain: StepData → BlobData with parent refs
const blobOptions = { allowMatureContent: this.allowMatureContent, ...options };
const blobOptions: BlobOptions = { allowMatureContent: this.allowMatureContent, ...options };
this.steps = (this.steps ?? []).map((rawStep: any) => {
if (rawStep instanceof StepData) {
rawStep._setWorkflow(this);
return rawStep;
}
return new StepData(rawStep, this.metadata, this, blobOptions);
return new StepData(rawStep, this, blobOptions);
});
}
@@ -78,24 +78,24 @@ export class WorkflowData {
return this.metadata?.remixOfId;
}
/** All succeeded, non-blocked, non-hidden images across all steps. */
get succeededImages(): BlobData[] {
return this.steps.flatMap((s) => s.succeededImages);
/** All succeeded, non-blocked, non-hidden outputs across all steps. */
get succeededOutput() {
return this.steps.flatMap((s) => s.succeededOutput);
}
/** All displayable images across all steps (includes upgradeable). */
get displayImages(): BlobData[] {
return this.steps.flatMap((s) => s.displayImages);
/** All displayable outputs across all steps (includes upgradeable). */
get displayOutput() {
return this.steps.flatMap((s) => s.displayOutput);
}
/** Total completed images across all steps. */
/** Total completed outputs across all steps. */
get completedCount() {
return this.steps.reduce((n, s) => n + s.completedCount, 0);
}
/** Total processing images across all steps. */
/** Total processing outputs across all steps. */
get processingCount() {
return this.steps.reduce((n, s) => n + s.processingCount, 0);
}
/** Total blocked images across all steps. */
/** Total blocked outputs across all steps. */
get blockedCount() {
return this.steps.reduce((n, s) => n + s.blockedCount, 0);
}
@@ -104,9 +104,9 @@ export class WorkflowData {
return this.steps.flatMap((s) => s.blockedReasons);
}
/** Create a StepData bound to this workflow's metadata. */
/** Create a StepData bound to this workflow. */
step(step: Record<string, any> & Pick<NormalizedStep, 'metadata'>) {
return new StepData(step, this.metadata, this);
return new StepData(step, this);
}
}
@@ -119,43 +119,36 @@ export class WorkflowData {
* getters that fall back to workflow metadata when step metadata is empty.
*/
export interface StepData extends NormalizedStep {
images: BlobData[];
output: Array<ImageBlob | VideoBlob | AudioBlob>;
}
export class StepData {
#wfMeta: NormalizedWorkflowMetadata | undefined;
#workflow: WorkflowData | undefined;
#workflow: WorkflowData;
constructor(
step: Record<string, any> & Pick<NormalizedStep, 'metadata'>,
wfMetadata?: NormalizedWorkflowMetadata,
workflow?: WorkflowData,
blobOptions?: WorkflowDataOptions & { allowMatureContent?: boolean | null }
workflow: WorkflowData,
blobOptions?: BlobOptions
) {
Object.assign(this, step);
this.#wfMeta = wfMetadata;
this.#workflow = workflow;
// Wrap raw images in BlobData when options are provided
// Wrap raw outputs in the appropriate BlobData subclass when options are provided.
// BlobData is abstract, so any instanceof BlobData is one of the concrete subclasses.
if (blobOptions) {
this.images = (this.images ?? ([] as any[])).map((img: any, index: number) =>
img instanceof BlobData
? img
: new BlobData({
data: img,
step: this,
index,
...blobOptions,
})
this.output = (this.output ?? ([] as any[])).map((item: any, index: number) =>
item instanceof BlobData
? (item as ImageBlob | VideoBlob | AudioBlob)
: BlobData.from(item, { step: this, index, ...blobOptions })
);
}
}
/** @internal Update parent workflow (used when re-parenting an existing StepData). */
/** @internal Re-parent onto a different WorkflowData (used when rebuilding WorkflowData from existing StepData instances). */
_setWorkflow(workflow: WorkflowData) {
this.#workflow = workflow;
}
get workflow(): WorkflowData | undefined {
get workflow(): WorkflowData {
return this.#workflow;
}
@@ -168,14 +161,14 @@ export class StepData {
if (stepParams && ('workflow' in stepParams || 'ecosystem' in stepParams)) {
return stepParams;
}
return this.#wfMeta?.params ?? stepParams ?? {};
return this.#workflow.metadata?.params ?? stepParams ?? {};
}
get resources(): NormalizedWorkflowMetadata['resources'] {
if (this.metadata.resources?.length) return this.metadata.resources;
return this.#wfMeta?.resources ?? [];
return this.#workflow.metadata?.resources ?? [];
}
get remixOfId() {
return this.metadata.remixOfId ?? this.#wfMeta?.remixOfId;
return this.metadata.remixOfId ?? this.#workflow.metadata?.remixOfId;
}
get prompt() {
return (this.params as Record<string, unknown>)?.prompt as string | undefined;
@@ -190,86 +183,77 @@ export class StepData {
return (this.metadata as any)?.suppressOutput === true;
}
/** Images that completed successfully, aren't blocked/moderated, and aren't hidden. */
get succeededImages(): BlobData[] {
/** Outputs that have landed, not blocked, and not hidden. */
get succeededOutput(): Array<ImageBlob | VideoBlob | AudioBlob> {
if (this.suppressOutput) return [];
return this.images.filter((x) => x.status === 'succeeded' && !x.blockedReason && !x.hidden);
return this.output.filter((x) => x.available && !x.blockedReason && !x.hidden);
}
/** Images suitable for display — not hidden, not hard-blocked (upgradeable images included). */
get displayImages(): BlobData[] {
/** Outputs suitable for display — not hidden, not hard-blocked (upgradeable + errored items included). */
get displayOutput(): Array<ImageBlob | VideoBlob | AudioBlob> {
if (this.suppressOutput) return [];
return this.images.filter((x) => x.displayable);
return this.output.filter((x) => x.displayable);
}
/** Count of images with status 'succeeded'. */
/** Count of outputs that have landed (available). */
get completedCount(): number {
return this.images.filter((x) => x.status === 'succeeded').length;
return this.output.filter((x) => x.available).length;
}
/** Count of images with status 'processing'. */
/** Count of outputs still waiting on a result (step hasn't terminated, blob not yet available, not blocked). */
get processingCount(): number {
return this.images.filter((x) => x.status === 'processing').length;
if (this.status && orchestratorCompletedStatuses.includes(this.status)) return 0;
return this.output.filter((x) => !x.available && !x.blockedReason).length;
}
/** Count of images with a blockedReason. */
/** Count of outputs with a blockedReason. */
get blockedCount(): number {
return this.images.filter((x) => !!x.blockedReason).length;
return this.output.filter((x) => !!x.blockedReason).length;
}
/** Blocked reason strings (for display grouping). */
get blockedReasons(): string[] {
return this.images.map((x) => x.blockedReason).filter((x): x is string => !!x);
return this.output.map((x) => x.blockedReason).filter((x): x is string => !!x);
}
}
// =============================================================================
// BlobData
// BlobData (abstract base)
// =============================================================================
type BlobConstructorArgs = {
data: NormalizedWorkflowStepOutput;
step: StepData;
index: number;
} & BlobOptions;
/**
* Image/video output with NSFW blocking logic and parent references.
* Extends NormalizedWorkflowStepOutput with `canUpgrade`, `step`, `workflow`.
* Abstract base for workflow output blobs. Concrete subclasses:
* - ImageBlob (type: 'image')
* - VideoBlob (type: 'video')
* - AudioBlob (type: 'audio')
*
* Subclasses carry no normalization logic everything is pre-shaped by
* `formatStepOutputs` before the raw payload reaches here. The base handles:
* - Shared blob fields (id, url, available, blockedReason, nsfwLevel, ...)
* - NSFW blocking / upgrade / private-gen rules
* - Parent-step ref and resolved metadata accessors (params/resources/remixOfId)
*/
export class BlobData implements NormalizedWorkflowStepOutput {
export abstract class BlobData {
abstract readonly type: 'image' | 'video' | 'audio';
url!: string;
workflowId!: string;
stepName!: string;
seed?: number | null;
status!: WorkflowStatus;
aspect!: number;
type!: 'image' | 'video';
id!: string;
available!: boolean;
urlExpiresAt?: string | null;
jobId?: string | null;
nsfwLevel?: NsfwLevel;
blockedReason?: string | null;
previewUrl?: string | null;
previewUrlExpiresAt?: string | null;
width!: number;
height!: number;
#step: StepData;
#index: number;
constructor({
data,
allowMatureContent,
step,
index,
domain,
nsfwEnabled,
}: {
data: ImageBlob | VideoBlob;
/** workflow.allowMatureContent */
allowMatureContent?: boolean | null;
step: StepData;
/** Position of this image within the step's images array. */
index: number;
domain: Record<ColorDomain, boolean>;
nsfwEnabled: boolean;
}) {
constructor({ data, allowMatureContent, step, index, domain, nsfwEnabled }: BlobConstructorArgs) {
Object.assign(this, data);
this.#step = step;
this.#index = index;
// Derive seed from step params base seed + image index
// Derive seed from step params base seed + output index
const baseSeed = step.params?.seed as number | undefined;
if (baseSeed != null) this.seed = baseSeed + index;
@@ -292,28 +276,78 @@ export class BlobData implements NormalizedWorkflowStepOutput {
}
}
/** Factory: instantiate the correct subclass based on `data.type`. */
static from(
data: NormalizedWorkflowStepOutput,
opts: Omit<BlobConstructorArgs, 'data'>
): ImageBlob | VideoBlob | AudioBlob {
const args = { data, ...opts } as BlobConstructorArgs;
switch (data.type) {
case 'image':
return new ImageBlob(args as BlobConstructorArgs & { data: NormalizedImageOutput });
case 'video':
return new VideoBlob(args as BlobConstructorArgs & { data: NormalizedVideoOutput });
case 'audio':
return new AudioBlob(args as BlobConstructorArgs & { data: NormalizedAudioOutput });
default: {
const _exhaustive: never = data;
void _exhaustive;
throw new Error('Unknown blob type');
}
}
}
get canUpgrade() {
return this.blockedReason === 'canUpgrade';
}
/** Whether this image should be shown in the UI (not hidden, not hard-blocked). */
/**
* Whether the output failed to materialize. True when the parent step reached a
* terminal state (`succeeded` / `failed` / `expired` / `canceled`) but the blob
* itself never became `available` indicates the worker finished without producing
* a usable output (e.g. blob upload failed post-job).
*/
get errored(): boolean {
if (this.available) return false;
const status = this.#step.status;
return !!status && orchestratorCompletedStatuses.includes(status);
}
/** Whether this output should be shown in the UI (not hidden, not hard-blocked). */
get displayable(): boolean {
return !this.hidden && (!this.blockedReason || this.canUpgrade);
}
/** Per-image metadata from step (hidden, feedback, favorite, etc.). */
get imageMeta() {
return (this.#step.metadata as any)?.images?.[this.id] as
| { hidden?: boolean; feedback?: 'liked' | 'disliked'; favorite?: boolean }
| undefined;
/**
* Per-output metadata from step (hidden, feedback, favorite, etc.).
*
* Merges the current `metadata.output` field with the legacy `metadata.images`
* field (pre-rename workflows). The new-key value wins per-field for a given
* blob id, so post-rename writes override legacy state cleanly while still
* preserving any legacy fields that haven't been re-written.
*/
get outputMeta():
| {
hidden?: boolean;
feedback?: 'liked' | 'disliked';
favorite?: boolean;
comments?: string;
postId?: number;
}
| undefined {
const meta = this.#step.metadata as any;
const legacy = meta?.images?.[this.id];
const current = meta?.output?.[this.id];
if (!legacy && !current) return undefined;
return { ...legacy, ...current };
}
/** Whether the user has marked this image as hidden (deleted). */
/** Whether the user has marked this output as hidden (deleted). */
get hidden(): boolean {
return this.imageMeta?.hidden ?? false;
return this.outputMeta?.hidden ?? false;
}
/** Position of this image within the step's images array. */
/** Position of this output within the step's output array. */
get index(): number {
return this.#index;
}
@@ -325,7 +359,17 @@ export class BlobData implements NormalizedWorkflowStepOutput {
/** Parent workflow. */
get workflow(): WorkflowData {
return this.step.workflow!;
return this.step.workflow;
}
/** Parent step name (derived from the parent StepData). */
get stepName(): string {
return this.#step.name;
}
/** Parent workflow id (derived from the parent WorkflowData). */
get workflowId(): string {
return this.workflow.id;
}
/** Resolved params from the parent step (step metadata → workflow metadata fallback). */
@@ -348,3 +392,38 @@ export class BlobData implements NormalizedWorkflowStepOutput {
return (this.params as any)?.ecosystem ?? this.params?.baseModel;
}
}
// =============================================================================
// Concrete blob subclasses
// =============================================================================
export class ImageBlob extends BlobData {
readonly type = 'image' as const;
width!: number;
height!: number;
aspect!: number;
previewUrl?: string | null;
previewUrlExpiresAt?: string | null;
constructor(args: BlobConstructorArgs & { data: NormalizedImageOutput }) {
super(args);
}
}
export class VideoBlob extends BlobData {
readonly type = 'video' as const;
width!: number;
height!: number;
aspect!: number;
constructor(args: BlobConstructorArgs & { data: NormalizedVideoOutput }) {
super(args);
}
}
export class AudioBlob extends BlobData {
readonly type = 'audio' as const;
readonly aspect = 1;
duration?: number | null;
constructor(args: BlobConstructorArgs & { data: NormalizedAudioOutput }) {
super(args);
}
}