fix(generation): resolve step-vs-workflow params via partialParams flag

Remix from an enhancement output (upscale, remove-bg) leaked the enhancement form's fields (images, upscaler, the img2img:* workflow key) into a remix of the original, because StepData.params merged step params over workflow params. Make resolution either/or by default (step params verbatim when present, else workflow fallback) so complete snapshots never leak.

Wildcard/snippet variants are the one true partial-delta case: the server stores the small delta in step params + sets a partialParams flag (passed through formatStep, not pre-merged), and the client spreads it over workflow params. This keeps the decision server-normalized while shipping only the small delta over the API. Replaces the earlier sourceLineage approach (and removes it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

docs/generation-metadata-architecture.md

@@ -0,0 +1,114 @@
+# Step vs. workflow metadata: overwrite-or-merge
+
+**Status:** Decided / implemented
+**Related:** [docs/features/legacy-metadata-mapping.md](features/legacy-metadata-mapping.md), [docs/workflow-metadata-refactor.md](workflow-metadata-refactor.md)
+
+> How the in-app generator decides whether a step's `params` replace or merge with the
+> workflow-level form snapshot when reconstructing a generation for remix.
+
+---
+
+## Scope (what this is and isn't)
+
+This doc is **only** about the in-app remix path, where generation data is split across two
+layers on a live workflow object:
+
+- `workflow.metadata.params` — the form-input snapshot for the whole submission.
+- `step.metadata.params` — per-step data.
+
+**Out of scope:** the EXIF `imageMetadata` we embed in generated images. That payload is already
+a flat, correct, self-contained snapshot — remixing from an uploaded image reads it directly and
+needs none of the two-layer reconciliation below. (We are explicitly **not** building a versioned
+canonical-snapshot format, `kind` enum, AIR-vs-id scheme, etc. — `imageMetadata` doesn't have the
+problem those would solve.)
+
+---
+
+## The problem
+
+When you remix a generated output in-app, `BlobData.params` → `StepData.params` resolves the
+output's params from the two layers. The question is what to do when the step carries its own
+`params`:
+
+- **Replace** (use step params verbatim), or
+- **Merge** (layer step params over `workflow.metadata.params`)?
+
+Getting this wrong is what caused the enhancement-remix bug (see
+[workflow-metadata-refactor.md](workflow-metadata-refactor.md)): an upscale/remove-bg step stores
+the **source generation's** params on the step, and `workflow.metadata.params` is the **enhancement
+form** (`images:[sourceUrl]`, `upscaler`, `img2img:upscale`, …). Merging leaked those enhancement
+fields into a remix of the original, so the remix behaved like the enhancement workflow.
+
+---
+
+## The rule: server flags the delta, client spreads it
+
+`step.metadata.params` is one of three things, and a server-set flag — `partialParams` — tells the
+client which, so the client never has to *decide*:
+
+| Step kind | `step.metadata.params` | `partialParams` | Resolution |
+| --------- | ---------------------- | --------------- | ---------- |
+| Standard generation | absent (data lives on `workflow.metadata`) | — | fall back to workflow |
+| Enhancement (upscale, remove-bg) | a **complete** snapshot of the source generation | — | use verbatim |
+| Wildcard/snippet variant | a small **delta** (substituted prompt) | **`true`** | spread over workflow params |
+
+`StepData.params` (in [workflow-data.ts](../src/shared/orchestrator/workflow-data.ts)) just applies
+the flag:
+
+```ts
+const stepParams = this.metadata.params;
+const wfParams = this.#workflow.metadata?.params;
+// partial delta (flagged by server): spread the small delta over the workflow form snapshot
+if (this.metadata.partialParams && stepParams && Object.keys(stepParams).length > 0) {
+  return { ...wfParams, ...stepParams };
+}
+// otherwise either/or: complete snapshot verbatim, or workflow fallback
+if (stepParams && Object.keys(stepParams).length > 0) return stepParams;
+return wfParams ?? {};
+```
+
+### Why this split — and why the spread is on the client
+
+- **The decision is server-side.** "Is this a partial delta?" is normalized into the `partialParams`
+  flag by the server (set at the write site that produces the delta, passed through `formatStep`).
+  The client contains no heuristic — it mechanically applies the flag.
+- **The spread is client-side, on purpose.** If the server pre-merged, it would send a full copy of
+  the params on *every* variant step — bloating the API response. Instead it sends the tiny delta +
+  the flag, and the client completes it against `workflow.metadata.params` (which it already has).
+- **`params` is never silently overloaded.** Without the flag, a step with params is a complete
+  snapshot (used verbatim) — so an enhancement remix never leaks the enhancement form's fields
+  (`images:[sourceUrl]`, `upscaler`, the `img2img:*` workflow key) into a remix of the original.
+
+---
+
+## Wildcards: the partial-delta case (plumbing implemented)
+
+Snippet/wildcard variants store *only* the substituted fields (e.g. `prompt`/`negativePrompt`) per
+step and rely on `workflow.metadata.params` for the full settings — a genuine partial **delta**. (The
+wildcards *UI feature* isn't shipped yet, but the read/merge path below is wired so it Just Works once
+wildcard generations are produced.)
+
+- **Write** ([orchestration-new.service.ts](../src/server/services/orchestrator/orchestration-new.service.ts),
+  snippet-variant loop): the per-variant overlay is written to `step.metadata.params` and the step is
+  flagged `partialParams: true`.
+- **Normalize** (`formatStep`): the delta is passed through **raw** (no `mapDataToGraphInput` — mapping
+  a bare prompt delta would fabricate a workflow/ecosystem key), and `partialParams` is forwarded to
+  `NormalizedStepMetadata`. The server does **not** pre-merge — it keeps the payload small.
+- **Read** (`StepData.params`): spreads the delta over `workflow.metadata.params` (see the rule above).
+
+Tradeoff: a partial step that *didn't* get flagged would be treated as a complete snapshot and used
+verbatim, dropping the workflow settings — so the write path must set `partialParams` whenever it
+writes a delta. We own both sides.
+
+---
+
+## History
+
+This started as a `sourceLineage` boolean (server marks "complete snapshot → don't merge", with merge
+as the *default*) — backwards polarity. We inverted it: **verbatim is the default**, and the rare
+partial-delta producer (wildcards) flags itself with `partialParams: true`. We briefly tried a
+dedicated `wildcards` key instead of a flag, but settled on the flag + `params` so the server can send
+just the small delta and the client does the spread — minimizing the API payload while keeping the
+decision server-normalized. The dead "Model A" source helpers
+(`buildStepSource`/`resolveStepSource`) were removed in the same effort — see
+[workflow-metadata-refactor.md](workflow-metadata-refactor.md).

src/server/services/orchestrator/__tests__/workflow-metadata.test.ts

@@ -145,138 +145,13 @@ describe('two-layer metadata model', () => {
     expect(getStepResources(step, workflow)).toEqual([{ id: 123 }]);
   });
 
-  it('enhancement step: step has own params, workflow has form input', () => {
-    // Enhancement steps always have their own params (the enhancement action)
+  it('enhancement step: complete step params are used verbatim, no workflow leak', () => {
+    // Enhancement steps (upscale, remove-bg) store the SOURCE generation's complete params on
+    // the step; workflow.metadata holds the enhancement form input. Either/or returns the step
+    // params verbatim, so the upscale form's images/upscaler/workflow key never leak into a
+    // remix of the original. Mirrors the real EXIF-sourced case where the source params carry
+    // no `workflow` key, so a merge would have leaked `img2img:upscale`.
     const step = makeStep({
-      params: { upscaler: '4x-ultrasharp', creativity: 0.5 },
-      resources: [],
-    });
-    const workflow = makeWorkflow({
-      params: { upscaler: '4x-ultrasharp', creativity: 0.5, workflow: 'img2img:upscale' },
-      resources: [],
-    });
-
-    // getStepParams returns step's own params (not falling through)
-    expect(getStepParams(step, workflow)).toEqual({
-      upscaler: '4x-ultrasharp',
-      creativity: 0.5,
-    });
-  });
-
-  it('isPrivateGeneration lives on workflow metadata', () => {
-    const workflow = makeWorkflow({
-      params: { prompt: 'a cat' },
-      resources: [],
-      isPrivateGeneration: true,
-    });
-
-    expect(workflow.metadata?.isPrivateGeneration).toBe(true);
-  });
-});
-
-// =============================================================================
-// WorkflowData class
-// =============================================================================
-
-describe('WorkflowData', () => {
-  it('resolves params from workflow metadata', () => {
-    const workflow = makeWorkflow({
-      params: { prompt: 'a cat', steps: 30 },
-      resources: [],
-    });
-    const wf = new WorkflowData(workflow, defaultOptions);
-
-    expect(wf.params).toEqual({ prompt: 'a cat', steps: 30 });
-  });
-
-  it('resolves resources from workflow metadata', () => {
-    const workflow = makeWorkflow({
-      params: {},
-      resources: [{ id: 1, modelName: 'SD 1.5' }] as any,
-    });
-    const wf = new WorkflowData(workflow, defaultOptions);
-
-    expect(wf.resources).toEqual([{ id: 1, modelName: 'SD 1.5' }]);
-  });
-
-  it('resolves remixOfId from workflow metadata', () => {
-    const workflow = makeWorkflow({
-      params: {},
-      resources: [],
-      remixOfId: 42,
-    });
-    const wf = new WorkflowData(workflow, defaultOptions);
-
-    expect(wf.remixOfId).toBe(42);
-  });
-
-  it('returns defaults when workflow metadata is undefined', () => {
-    const workflow = makeWorkflow(undefined);
-    const wf = new WorkflowData(workflow, defaultOptions);
-
-    expect(wf.params).toEqual({});
-    expect(wf.resources).toEqual([]);
-    expect(wf.remixOfId).toBeUndefined();
-  });
-
-  it('step() creates StepData bound to workflow metadata', () => {
-    const workflow = makeWorkflow({
-      params: { prompt: 'wf prompt' },
-      resources: [{ id: 1 }] as any,
-      remixOfId: 42,
-    });
-    const step = makeStep({});
-    const wf = new WorkflowData(workflow, defaultOptions);
-    const sd = wf.step(step);
-
-    expect(sd.params).toEqual({ prompt: 'wf prompt' });
-    expect(sd.resources).toEqual([{ id: 1 }]);
-    expect(sd.remixOfId).toBe(42);
-  });
-
-  it('exposes underlying NormalizedWorkflow properties directly', () => {
-    const workflow = makeWorkflow({
-      params: { prompt: 'a cat' },
-      resources: [],
-    });
-    const wf = new WorkflowData(workflow, defaultOptions);
-
-    expect(wf.id).toBe('wf-1');
-    expect(wf.status).toBe('succeeded');
-    expect(wf.tags).toEqual([]);
-  });
-});
-
-// =============================================================================
-// StepData class
-// =============================================================================
-
-describe('StepData', () => {
-  it('returns step params when present', () => {
-    const step = makeStep({ params: { prompt: 'step prompt' } });
-    const wf = makeWorkflowData({ params: { prompt: 'wf prompt' }, resources: [] });
-    const sd = new StepData(step, wf);
-
-    expect(sd.params).toEqual({ prompt: 'step prompt' });
-  });
-
-  it('falls back to workflow metadata when step has no params', () => {
-    const step = makeStep({});
-    const wf = makeWorkflowData({ params: { prompt: 'wf prompt', steps: 20 }, resources: [] });
-    const sd = new StepData(step, wf);
-
-    expect(sd.params).toEqual({ prompt: 'wf prompt', steps: 20 });
-  });
-
-  it('enhancement step (sourceLineage): does NOT leak workflow form fields into source params', () => {
-    // Enhancement steps are flagged `sourceLineage: true` by the server and store the
-    // SOURCE generation's complete params; the workflow-level metadata is the upscale form
-    // input. The two must NOT merge — otherwise a remix of the upscaled output picks up the
-    // upscale form's `images`/`upscaler`/`workflow` and behaves like the enhancement
-    // workflow. Mirrors the real EXIF-sourced case where the source params carry no
-    // `workflow` key, so a merge would leak `img2img:upscale`.
-    const step = makeStep({
-      sourceLineage: true,
       params: { prompt: 'a cat', seed: 123, baseModel: 'SDXL' },
     });
     const wf = makeWorkflowData({
@@ -292,21 +167,20 @@ describe('StepData', () => {
     });
     const sd = new StepData(step, wf);
 
-    // Pure source params — none of the upscale form fields (incl. the workflow key) leak.
     expect(sd.params).toEqual({ prompt: 'a cat', seed: 123, baseModel: 'SDXL' });
   });
 
-  it('wildcard/snippet step (no sourceLineage): MERGES partial delta over workflow params', () => {
-    // Snippet variants store ONLY the substituted prompt fields on the step and rely on
-    // workflow.metadata.params for the full settings snapshot. Without sourceLineage the
-    // step delta must merge over the workflow params (step wins per-field).
+  it('partialParams (wildcard/snippet variant): spreads the params delta over workflow params', () => {
+    // Snippet variants store a small DELTA in `params` and set `partialParams: true`. The client
+    // spreads that delta over the workflow form snapshot (the server sends only the delta).
     const step = makeStep({
+      partialParams: true,
       params: { prompt: 'a substituted cat', negativePrompt: 'blurry' },
     });
     const wf = makeWorkflowData({
       params: {
         workflow: 'txt2img',
-        prompt: 'a #animal', // template prompt — overridden by the substituted step prompt
+        prompt: 'a #animal', // template prompt — overridden by the substituted delta prompt
         negativePrompt: 'lowres', // overridden too
         steps: 30,
         cfgScale: 7,
@@ -317,7 +191,6 @@ describe('StepData', () => {
     });
     const sd = new StepData(step, wf);
 
-    // Step delta wins on prompt/negativePrompt; all other settings come from workflow.
     expect(sd.params).toEqual({
       workflow: 'txt2img',
       prompt: 'a substituted cat',
@@ -329,6 +202,19 @@ describe('StepData', () => {
     });
   });
 
+  it('without partialParams, a step with params is used verbatim (no spread)', () => {
+    // Same shapes as above but no flag — the step params are treated as a complete snapshot and
+    // returned verbatim; the workflow settings are NOT merged in.
+    const step = makeStep({ params: { prompt: 'a substituted cat', negativePrompt: 'blurry' } });
+    const wf = makeWorkflowData({
+      params: { workflow: 'txt2img', steps: 30, cfgScale: 7 } as any,
+      resources: [],
+    });
+    const sd = new StepData(step, wf);
+
+    expect(sd.params).toEqual({ prompt: 'a substituted cat', negativePrompt: 'blurry' });
+  });
+
   it('returns step resources when present', () => {
     const step = makeStep({ resources: [{ id: 1 }] as any });
     const wf = makeWorkflowData({ params: {}, resources: [{ id: 2 }] as any });

src/server/services/orchestrator/orchestration-new.service.ts

@@ -986,10 +986,12 @@ export async function createWorkflowStepsFromGraph({
     );
 
     // Per-step metadata for snippet variants records ONLY the substituted
-    // snippet-target fields (e.g. `prompt`, `negativePrompt`) — the
-    // workflow-level metadata already carries the full template + params
-    // snapshot, so duplicating it on every step would just bloat storage.
-    // The overlay IS the per-step delta from the workflow params.
+    // snippet-target fields (e.g. `prompt`, `negativePrompt`) — a small DELTA — in
+    // `params`, plus `partialParams: true` to flag it as such. The workflow-level
+    // metadata already carries the full template + params snapshot, so we keep the
+    // per-step payload tiny (no full params duplicated per variant) and let the
+    // client spread the delta over the workflow params when reading.
+    // See docs/generation-metadata-architecture.md.
     if (isSnippetVariant) {
       for (const step of variantSteps) {
         const existingMeta = (step.metadata ?? {}) as Record<string, unknown>;
@@ -997,6 +999,7 @@ export async function createWorkflowStepsFromGraph({
         step.metadata = {
           ...existingMeta,
           params: { ...existingParams, ...overlay },
+          partialParams: true,
         };
       }
     }
@@ -1514,19 +1517,14 @@ export interface NormalizedStepMetadata {
    */
   params?: Partial<GenerationGraphValues> & Record<string, unknown>;
   /**
-   * When true, `params` is a COMPLETE, self-contained snapshot — the source generation of
-   * an enhancement step (upscale, remove-bg). Consumers (StepData.params) must use it
-   * verbatim, NOT merge it over workflow.metadata.params, or the enhancement form's fields
-   * (`images`, `upscaler`, its `workflow` key) leak into a remix of the original.
-   *
-   * Falsy/undefined means `params` is either absent (standard new-format gen → fall back to
-   * workflow.metadata.params) or a partial DELTA (wildcard/snippet variants store only the
-   * substituted prompt fields → merge over workflow.metadata.params).
-   *
-   * Derived per-step in formatStep from the raw step's lineage marker (`'workflow' in
-   * step.metadata`, plus the legacy `transformations`/`source` formats).
+   * When true, `params` is a partial DELTA (e.g. a wildcard/snippet variant's substituted
+   * `prompt`/`negativePrompt`) rather than a complete snapshot. The client (`StepData.params`)
+   * spreads it over `workflow.metadata.params`; the server passes the small delta through rather
+   * than pre-merging, to keep the API payload small (no full params duplicated per variant).
+   * Absent for complete-snapshot (enhancement) and standard steps. Set deliberately at the write
+   * site that produces the delta — not derived. See docs/generation-metadata-architecture.md.
    */
-  sourceLineage?: boolean;
+  partialParams?: boolean;
   /**
    * Source generation resources (for steps with source lineage).
    * Undefined for standard generation steps (use workflow.metadata.resources instead).
@@ -2047,16 +2045,14 @@ function formatStep(
   // via StepData.params. Running mapDataToGraphInput would double-resolve from
   // resources and inject a duplicate workflow key.
   const hasSourceLineage = 'workflow' in metadata;
-  // Whether the resolved step params are a COMPLETE source snapshot (enhancement step, or
-  // the legacy transformations/source formats) vs. a partial delta. Surfaced to the client
-  // as `metadata.sourceLineage` so StepData.params can pick verbatim-vs-merge.
-  const sourceLineage =
-    hasSourceLineage ||
-    (Array.isArray(transformations) && transformations.length > 0) ||
-    (metadata.source != null && typeof metadata.source === 'object');
+  // A partial-delta step (wildcard/snippet variant) carries only a small overlay (e.g. the
+  // substituted prompt). Pass it through raw — the client spreads it over workflow params.
+  // Don't run mapDataToGraphInput on a bare prompt delta: it would fabricate a workflow/
+  // ecosystem key from no context and pollute the delta.
+  const partialParams = (metadata as { partialParams?: boolean }).partialParams === true;
   let finalParams: Record<string, unknown> | undefined;
   if (resolvedParams) {
-    if (hasSourceLineage) {
+    if (partialParams || hasSourceLineage) {
       finalParams = removeEmpty(resolvedParams);
     } else {
       const mapped = mapDataToGraphInput(resolvedParams, resolvedResources ?? [], {
@@ -2085,8 +2081,6 @@ function formatStep(
     metadata: {
       ...removeEmpty({
         params: finalParams,
-        // Only emit when params is a complete snapshot — absent (falsy → merge) otherwise.
-        sourceLineage: sourceLineage && finalParams ? true : undefined,
         remixOfId,
         // Pass both raw keys through. Client merges `output + images` for display
         // via `BlobData.outputMeta`; client's patch builder inspects `output`
@@ -2096,6 +2090,9 @@ function formatStep(
         output: (metadata as any).output as NormalizedStepMetadata['output'],
         images: metadata.images as NormalizedStepMetadata['images'],
         suppressOutput: metadata.suppressOutput as boolean | undefined,
+        // Flag (not a pre-merge): tells the client to spread the small `params` delta over
+        // workflow params. Kept as a flag so the API payload stays minimal.
+        partialParams: partialParams && finalParams ? true : undefined,
       }),
       ...(resolvedResources?.length ? { resources: resolvedResources } : {}),
     },

src/shared/orchestrator/workflow-data.ts

@@ -156,27 +156,24 @@ export class StepData {
     const stepParams = this.metadata.params;
     const wfParams = this.#workflow.metadata?.params;
 
-    // `step.metadata.params` is overloaded with two opposite meanings, told apart by the
-    // server-set `sourceLineage` flag (a per-step fact `formatStep` derives from the raw
-    // step's lineage marker):
+    // The server flags whether `step.metadata.params` is a partial DELTA vs a complete snapshot,
+    // so this getter never has to *decide* — it just applies the directive:
     //
-    // 1. sourceLineage === true (enhancement steps — upscale, remove-bg):
-    //    params is a COMPLETE, self-contained snapshot of the *source* generation. Use it
-    //    verbatim. Merging in workflow.metadata.params (the enhancement form input) would
-    //    leak fields the source doesn't override — e.g. `images:[sourceUrl]`, `upscaler`,
-    //    or the `img2img:upscale` workflow key — into a remix of the original, making the
-    //    remix behave like the enhancement workflow.
+    // - `partialParams` set (wildcard/snippet variant): `params` is a small overlay (e.g. the
+    //   substituted prompt). Spread it over the workflow-level form snapshot to reconstruct the
+    //   variant's effective params. The server sends only the delta — we complete it here, which
+    //   keeps the API payload small (no full params duplicated per variant).
+    // - otherwise: either/or. A complete snapshot (enhancement steps store the *source*
+    //   generation here) is used verbatim; an absent one falls back to workflow params. We must
+    //   NOT spread in this case, or the enhancement form's fields (`images`, `upscaler`, the
+    //   `img2img:*` workflow key) would leak into a remix of the original.
     //
-    // 2. sourceLineage falsy (wildcard/snippet variants):
-    //    params is a partial DELTA (only the substituted `prompt`/`negativePrompt`). It
-    //    MUST merge over workflow.metadata.params, which holds the full template + settings
-    //    snapshot (steps, cfgScale, sampler, seed, resources, workflow key).
-    //
-    // Empty/absent step params (standard new-format gen) fall back to workflow params.
-    if (this.metadata.sourceLineage && stepParams && Object.keys(stepParams).length > 0) {
-      return stepParams;
+    // See docs/generation-metadata-architecture.md.
+    if (this.metadata.partialParams && stepParams && Object.keys(stepParams).length > 0) {
+      return { ...wfParams, ...stepParams };
     }
-    return { ...wfParams, ...stepParams };
+    if (stepParams && Object.keys(stepParams).length > 0) return stepParams;
+    return wfParams ?? {};
   }
   get resources(): NormalizedWorkflowMetadata['resources'] {
     if (this.metadata.resources?.length) return this.metadata.resources;