From fdd6457fed4d00aef47e40cc0a7635e6d37d5729 Mon Sep 17 00:00:00 2001 From: calesthio Date: Tue, 28 Apr 2026 08:11:31 -0700 Subject: [PATCH] docs(prompting): adopt 5-aspect video specification across skills Incorporate the structured taxonomy from Lin et al. "Building a Precise Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2). The paper proves prompts structured around five aspects (Subject / Subject Motion / Scene / Spatial Framing / Camera) unlock controllable cinematography in fine-tuned video generation models. Off-the-shelf VLMs already nail subject and scene; the gains live in motion, spatial, and camera, which prompts routinely omit. Universal layer (skills/creative/video-gen-prompting.md, +125 lines): - 5-aspect prompt skeleton replaces flat formula - Camera movements regrouped (translation / rotation / lens-only) with dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations - New primitive tables: camera height, camera angle, POV, lens distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking), playback speed (6 modes), subject transitions - Order-matters and self-contained-prompt rules - Identity anchoring rule for multi-shot - Strict static-shot rule, anti-subjective callout, overlays-not-depth - Per-model word-count guidance Per-model guides (sora, veo, hunyuan, ltx, seedance): - Add the primitives each model honors literally - Word-count sweet spots per model - Strengthen seedance verbatim-identity and subject-transition guidance Pipeline directors (cinematic / explainer / animation scene-director, cinematic / explainer asset-director): - 5-aspect scene-plan checklist (per-pipeline adapted) - Overlays-not-depth callout - Pre / critique / post self-review loop for generation prompts Reviewer (skills/meta/reviewer.md): - CHAI critique-quality rules: accurate / complete / constructive - Critical findings now require a proposed_fix Storytelling, cinematic, broll, video-reference-analyst: - Anti-subjective rule (replace mood adjectives with visual causes) - Camera-intent-per-beat for script writers - POV column in stock-footage query templates - 5-aspect structured output mandatory for reference-video analysis skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec. --- skills/INDEX.md | 4 +- skills/creative/broll-planning.md | 19 +- skills/creative/cinematic.md | 16 ++ .../creative/prompting/hunyuan-prompting.md | 29 ++- skills/creative/prompting/ltx-prompting.md | 7 +- .../creative/prompting/seedance-prompting.md | 19 ++ skills/creative/prompting/sora-prompting.md | 5 + skills/creative/prompting/veo-prompting.md | 20 +- skills/creative/storytelling.md | 40 ++++ skills/creative/video-gen-prompting.md | 178 +++++++++++++++--- skills/meta/reviewer.md | 19 +- skills/meta/video-reference-analyst.md | 29 ++- skills/pipelines/animation/scene-director.md | 18 +- skills/pipelines/cinematic/asset-director.md | 16 ++ skills/pipelines/cinematic/scene-director.md | 19 +- skills/pipelines/explainer/asset-director.md | 16 ++ skills/pipelines/explainer/scene-director.md | 14 ++ 17 files changed, 418 insertions(+), 50 deletions(-) diff --git a/skills/INDEX.md b/skills/INDEX.md index 15a297eb..02767cf0 100644 --- a/skills/INDEX.md +++ b/skills/INDEX.md @@ -58,6 +58,7 @@ Key capability families to look for in the output: | `audio_processing` | — | FFmpeg-based local tools | | `enhancement` | — | Mixed providers | | `analysis` | — | Mixed providers | +| `character_animation` | — | Local character specs, SVG rigs, pose libraries, action timelines, previews, and QA | | `graphics` | — | Local rendering tools | | `music_generation` | — | Single-provider | | `subtitle` | — | Pure Python | @@ -92,7 +93,7 @@ Key capability families to look for in the output: | Enhancement Strategy | `creative/enhancement-strategy.md` | Overlay placement and density | `ffmpeg` | | Data Visualization | `creative/data-visualization.md` | Chart type selection, animation, label placement | `d3-viz`, `remotion-best-practices` | | Video Stitching | `creative/video-stitching.md` | Multi-clip assembly, AI clip chaining, spatial composition | `ffmpeg`, `video_toolkit` | -| Video Gen Prompting | `creative/video-gen-prompting.md` | Universal video generation prompt vocabulary | `ai-video-gen`, `ltx2`, `create-video` | +| Video Gen Prompting | `creative/video-gen-prompting.md` | Universal video generation prompt vocabulary; **canonical 5-aspect spec** (Subject / Motion / Scene / Spatial / Camera); ~200 cinematography primitives | `ai-video-gen`, `ltx2`, `create-video` | | ↳ Seedance Prompting | `creative/prompting/seedance-prompting.md` | **Preferred premium default.** Seedance 2.0 8-component structure, multi-shot, lip-sync, reference-to-video | `seedance-2-0`, `ai-video-gen` | | ↳ Grok Prompting | `creative/prompting/grok-prompting.md` | Grok image/video prompting, edit flows, reference-image video | `grok-media` | | ↳ Sora Prompting | `creative/prompting/sora-prompting.md` | Sora 2 structured template, advanced fields | `ai-video-gen` | @@ -127,6 +128,7 @@ Pipeline type skills provide production guidance for specific video formats, ind | Long-Form | `creative/long-form.md` | YouTube 10+ min â€" chapters, retention, end screens | | Screen Recording | `creative/screen-recording.md` | Code walkthroughs, tutorials, software demos | | Animation Pipeline | `creative/animation-pipeline.md` | Motion graphics, easing, transitions, composition | +| Character Animation Pipeline | `pipelines/character-animation/` | Rigged local cartoon characters, pose libraries, action timelines, SVG/Canvas/Remotion/HyperFrames rendering | | Cinematic | `creative/cinematic.md` | Letterbox, film pacing, layered audio, color grading | ## Pipeline Stage Director Skills diff --git a/skills/creative/broll-planning.md b/skills/creative/broll-planning.md index 13bf5fc7..3f44ecb8 100644 --- a/skills/creative/broll-planning.md +++ b/skills/creative/broll-planning.md @@ -70,14 +70,18 @@ Fallback: AI-generated image of server racks ### Query Templates by Scene Type -| Scene Type | Query Template | Example | +Add a **POV keyword** to every query. Stock libraries (Pexels, Pixabay, Storyblocks, Artgrid) explicitly index POV terms — drone, aerial, OTS (over-the-shoulder), macro, top-down, dashcam, FPV, handheld, locked-off — and adding the POV often unlocks better matches than refining the subject. The CMU/Harvard CHAI taxonomy treats POV as a first-class Scene aspect for the same reason: it changes which library shelf you're searching. + +| Scene Type | Query Template | Example with POV | |-----------|---------------|---------| -| Establishing | `[place] [time of day]` | "tokyo skyline night" | -| Activity | `[person] [action]` | "scientist microscope" | -| Object | `[object] [style]` | "circuit board closeup" | -| Nature | `[element] [quality]` | "ocean waves aerial" | -| Abstract motion | `[movement] [style]` | "light trails timelapse" | -| Workplace | `[setting] [activity]` | "modern office meeting" | +| Establishing | `[place] [time of day] [POV]` | "tokyo skyline night drone" | +| Activity | `[person] [action] [POV]` | "scientist microscope OTS" | +| Object | `[object] [style] [POV]` | "circuit board macro top-down" | +| Nature | `[element] [quality] [POV]` | "ocean waves aerial drone" | +| Abstract motion | `[movement] [style] [POV]` | "light trails timelapse locked-off" | +| Workplace | `[setting] [activity] [POV]` | "modern office meeting handheld" | + +If the scene description doesn't already imply a POV, ask the script/scene director — don't default to "no POV." A wrong-POV match (handheld when the scene needs drone) is harder to fix than a wrong color grade. ## Evaluating Stock Footage Quality @@ -89,6 +93,7 @@ When the stock tool returns results, evaluate before using: - **Style compatibility:** Doesn't clash with the playbook's visual style - **No watermarks:** Pexels/Pixabay are license-free, but verify - **Composition:** Subject is well-framed, not cut off awkwardly +- **POV match:** Does the footage's actual POV (drone, OTS, macro, handheld, locked-off, etc.) match what the scene needs? A wrong POV — e.g., handheld when the scene wants drone — is **more costly to fix than a wrong color grade**. Reject and re-query rather than try to crop your way out of it. ### Video Criteria (all image criteria plus) - **Duration:** At least as long as the scene needs (can trim, can't extend) diff --git a/skills/creative/cinematic.md b/skills/creative/cinematic.md index e1e4b8d0..dd26ded2 100644 --- a/skills/creative/cinematic.md +++ b/skills/creative/cinematic.md @@ -3,6 +3,8 @@ > Sources: No Film School editorial guides, StudioBinder filmmaking resources, Film Riot > production tutorials, CinematographyDB shot databases, Walter Murch "In the Blink of an Eye" +> For the universal cinematography vocabulary (camera, lens, motion, focus primitives, plus the 5-aspect Subject / Subject Motion / Scene / Spatial Framing / Camera spec), see `skills/creative/video-gen-prompting.md`. This file layers cinematic-specific conventions on top of those primitives — it does not redefine them. + ## Quick Reference Card ``` @@ -16,6 +18,19 @@ MUSIC: 60-90 BPM, orchestral or ambient, dynamic (not loop-based) TARGET LUFS: -14 LUFS integrated, -24 LUFS for quiet moments ``` +## Replace Mood Adjectives with Visual Causes + +> **"Cinematic" and "epic" don't constrain pixels.** The cinematic look comes from concrete choices: aspect ratio, lens, lighting key, color grade, shot duration, and audio layer count. State those — the rest is decoration. +> +> The CMU/Harvard CHAI study showed subjective phrasing varies wildly across annotators and model interpretations, which means a beat tagged "moody" routes to a different visual every render. Replace the adjective with the lighting + grade + shot-duration combination that produces moodiness. See `skills/creative/storytelling.md` "Anti-Subjective Rule" for the script-side equivalent. +> +> | Mood adjective | Cinematic translation | +> |---|---| +> | "epic" | 2.39:1 letterbox, 24fps, 8s+ shot duration, orchestral score with crescendo | +> | "moody" | `moody_dark` grade at 0.6, key light at 1/8 fill, 6s+ contemplative shots, ambient bed at -28dB | +> | "intimate" | 1.85:1, 40-50mm equivalent, shallow DoF, 2 audio layers (dialogue + room tone), no music under dialogue | +> | "cinematic" | (this word is banned — pick one of the above or describe the actual choices) | + ## Aspect Ratios | Ratio | Resolution (in 1080p frame) | Feel | When to Use | @@ -113,6 +128,7 @@ Add a subtle ambient layer to fill silence and create depth: - Highlights should be slightly rolled off (never pure white) - Skin tones must stay on the vectorscope skin tone line - Consistency across all clips — one LUT/profile for the entire video +- **If a beat is described as "moody," rewrite as the lighting + grade + shot-duration combination that produces moodiness.** Don't pass mood adjectives to the asset/edit stages. ## Applying to OpenMontage diff --git a/skills/creative/prompting/hunyuan-prompting.md b/skills/creative/prompting/hunyuan-prompting.md index 04ad5a35..dd385793 100644 --- a/skills/creative/prompting/hunyuan-prompting.md +++ b/skills/creative/prompting/hunyuan-prompting.md @@ -3,6 +3,8 @@ > Source: [Tencent Prompt Handbook](https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/main/assets/HunyuanVideo_1_5_Prompt_Handbook_EN.md) > For universal vocabulary, see: `skills/creative/video-gen-prompting.md` +**Word count:** Hunyuan 1.5 reads well at 80–200 words; doesn't reward 400-word essays. + ## HunyuanVideo Prompt Formula ### Text-to-Video @@ -34,15 +36,28 @@ Describe lighting with multiple dimensions: | Movement | Type | HunyuanVideo Prompt | |----------|------|-------------------| -| Crane / Pedestal | Vertical | "camera rises vertically" | -| Truck / Tracking | Horizontal | "camera tracks left alongside subject" | -| Dolly In | Push | "camera pushes forward toward subject" | -| Dolly Out | Pull | "camera pulls back from subject" | -| Pan | Rotation | "camera pans right across the scene" | +| Crane / Pedestal | Translation (vertical) | "camera rises vertically" | +| Truck / Tracking | Translation (horizontal) | "camera tracks left alongside subject" | +| Dolly In | Translation (push) | "camera pushes forward toward subject" | +| Dolly Out | Translation (pull) | "camera pulls back from subject" | +| Pan | Rotation (yaw) | "camera pans right across the scene" | +| Tilt | Rotation (pitch) | "camera tilts upward to follow the rocket" | +| Roll | Rotation (Z-axis / Dutch) | "camera rolls clockwise into a Dutch tilt" | | Orbit | Circular | "camera orbits around subject" | | Follow | Lock-on | "camera follows subject from behind" | +| Zoom | Lens-only (focal length) | "camera slowly zooms in on the figure" | +| Rack focus | Lens-only (focal plane, snap) | "rack focus from the foreground bottle to the figure in the background" | +| Pull focus | Lens-only (focal plane, gradual) | "camera shifts focus from foreground X to background Y" | | Static | Fixed | "static camera, no movement" | +### Focal-plane labels at start AND end of dynamic-DoF shots + +Hunyuan benefits when both endpoints of focus-changing shots are stated. State where focus starts AND where it lands — don't leave one implicit. + +Example: "shallow DoF; focus on the foreground bottle at start; focus pulls to the figure in the background by end." + +Without both endpoints, Hunyuan often defaults to deep focus or holds on the wrong plane. + ### Style Keywords **Photorealistic / Cinematic**: @@ -65,6 +80,10 @@ When using image-to-video, the input image defines appearance. Your prompt shoul **Bad I2V prompt**: "A beautiful woman in a red dress standing in a forest" — this repeats what the image already shows. +### Order motions temporally + +Describe motion in temporal order; if multiple movements occur, separate them ("first the camera pans right, then tilts upward"). Hunyuan executes motion in the order it appears in the prompt — bundling two movements into one clause causes one of them to be dropped or blended. + ## Example (T2V) ``` diff --git a/skills/creative/prompting/ltx-prompting.md b/skills/creative/prompting/ltx-prompting.md index b74d6713..6606b9eb 100644 --- a/skills/creative/prompting/ltx-prompting.md +++ b/skills/creative/prompting/ltx-prompting.md @@ -11,9 +11,13 @@ LTX-2 uses a clean, focused prompt structure: 2. **Set the scene** — lighting, color palette, textures, atmosphere 3. **Describe the action** — natural sequence flowing from beginning to end 4. **Define the character(s)** — physical cues (age, hair, clothes), not abstract labels -5. **Camera movement(s)** — specify how and when; describe what appears AFTER the movement +5. **Camera movement(s)** — specify how and when; describe what appears AFTER the movement. (LTX honors the translation/rotation/lens distinction: `dolly` ≠ `zoom`, `pan` ≠ `truck`. Pick the right family — translation moves the rig, rotation pivots it, lens-only changes focal length or focal plane without moving the camera.) 6. **Describe the audio** — ambient sound, music, speech, or singing +### Strict-Static-Shot rule + +If you write "static camera," the shot must have NO movement, NO focus change, NO zoom. LTX takes "static" literally — adding any motion verb later in the prompt will either be ignored or will produce a glitch where the camera contradicts itself. Pick one: static, OR a single named movement. + ## LTX-Specific Tips ### Post-Movement Description @@ -51,6 +55,7 @@ LTX organizes styles into three families: | Overloaded scenes | Many characters/actions reduces coherence | | Conflicting lighting descriptions | Pick one setup, commit to it | | Starting complex | Build up: simple prompt first, add layers | +| Prompts over ~80 words | LTX-2 degrades past that. Pick the most important 5–6 elements. | ## LTX Technical Notes diff --git a/skills/creative/prompting/seedance-prompting.md b/skills/creative/prompting/seedance-prompting.md index 9bc8a2ef..657ebd83 100644 --- a/skills/creative/prompting/seedance-prompting.md +++ b/skills/creative/prompting/seedance-prompting.md @@ -44,6 +44,8 @@ Seedance is unusually literal about camera language, multi-shot cuts, and quoted ## Multi-shot pattern +> **Repeat identity verbatim across every shot.** "the same character" / pronouns / "Aang again" do not work. Repeat the 3–6 disambiguating visual attributes verbatim in every shot block. Seedance treats each shot as if you said it cold. + Seedance honors explicit shot lists: ``` @@ -63,6 +65,22 @@ Style: anamorphic lens, teal-orange cinematic grade, 35mm film grain. Audio: rising orchestral swell with low taiko pulse, wind, distant wingbeats. ``` +### Subject transition primitives in multi-shot + +Seedance handles four distinct ways a subject can enter or exit a shot. Naming the primitive explicitly helps the model build the right transition between shots. + +- **Subject revealing** (by camera move OR subject move) — the subject becomes visible mid-shot. + Example: `Shot 2 (slow truck right): empty corridor at first; the camera trucks right to reveal Aang — bald, blue arrow tattoo, orange robes — pressed flat against the wall.` + +- **Subject disappearing** — the subject leaves frame, by motion or occlusion. + Example: `Shot 4 (static wide): Aang — bald, blue arrow tattoo, orange robes — sprints into the temple doorway and is swallowed by shadow; camera holds on the empty threshold.` + +- **Subject switching** (rack focus / camera move) — focus or framing transfers from one subject to another. + Example: `Shot 5 (close-up, rack focus): rack focus from Aang's glowing arrow tattoo in foreground to Sokka — dark hair, blue tunic, boomerang on back — emerging from the mist behind.` + +- **Complex alternating focus** — focus oscillates between two subjects within one shot. + Example: `Shot 7 (medium two-shot, alternating rack focus): focus on Aang — bald, blue arrow tattoo, orange robes — as he speaks, then pulls to Katara — long brown hair, blue water-tribe parka — as she answers, then back to Aang on the final beat.` + ## Lip-sync pattern ``` @@ -84,6 +102,7 @@ Sokka, half a step behind, replies: "Then we fight." | `generate_audio` | Keep `true` — sync audio is the moat. Strip in compose if unused. | | `model_variant` | `standard` for hero + multi-shot + camera-heavy. `fast` for b-roll, previews, latency-capped jobs. | | `seed` | Lock once a shot composition reads; iterate variants with the same seed. | +| `prompt length` | 200–400 words for hero shots; 80–150 for inserts. Seedance is one of the few models that rewards long, structured 5-aspect prompts. | ## Iteration strategy diff --git a/skills/creative/prompting/sora-prompting.md b/skills/creative/prompting/sora-prompting.md index fbd738fa..4e0f3d4f 100644 --- a/skills/creative/prompting/sora-prompting.md +++ b/skills/creative/prompting/sora-prompting.md @@ -3,6 +3,8 @@ > Source: [OpenAI Sora 2 Cookbook](https://developers.openai.com/cookbook/examples/sora/sora2_prompting_guide) > For universal vocabulary, see: `skills/creative/video-gen-prompting.md` +**Word count:** Sora 2 plateaus around 100–250 words. Past 250, additional detail rarely improves output. + ## Sora-Specific Prompt Template Sora responds best to a structured format with prose + cinematography block + action beats: @@ -40,6 +42,9 @@ Sora uniquely responds to these production-level details that most models ignore | **Wardrobe** | "navy coat, sleeves rolled, suspenders loose" | | **Finishing** | "fine-grain overlay, mild halation, gate weave, soft vignette" | | **Shutter** | "180° shutter angle" | +| **Playback speed** | "speed ramp from 1x to 0.25x mid-shot", "stop-motion staccato", "time-reversed exhale" | +| **Lens distortion** | "fisheye barrel distortion at the edges", "subtle barrel curvature on straight lines" | +| **Focus mode** | "rack focus from foreground bottle to background figure", "deep focus, FG to BG sharp" | ## What Sora Does Differently diff --git a/skills/creative/prompting/veo-prompting.md b/skills/creative/prompting/veo-prompting.md index 791815b9..c9593ad2 100644 --- a/skills/creative/prompting/veo-prompting.md +++ b/skills/creative/prompting/veo-prompting.md @@ -3,6 +3,8 @@ > Source: [Vertex AI Video Gen Prompt Guide](https://cloud.google.com/vertex-ai/generative-ai/docs/video/video-gen-prompt-guide) > For universal vocabulary, see: `skills/creative/video-gen-prompting.md` +**Word count:** VEO 3.1 sweet spot is 100–250 words; longer prompts stop helping. + ## VEO-Specific 14-Component Structure VEO responds to the most comprehensive prompt structure of any model: @@ -29,16 +31,30 @@ VEO responds to the most comprehensive prompt structure of any model: - **Negative prompts**: Explicitly supported — "no text overlays, no watermarks, no lens flare" - **Editing vocabulary**: Understands "match cut", "jump cut", "montage", "split diopter" as prompt terms. +### Camera vocabulary VEO honors literally + +VEO 3.1 distinguishes the three camera-motion families and treats their tokens as separate primitives. Mixing them up (e.g. asking for a "zoom" when you mean a "dolly") will produce the wrong move. + +- **Translation (rig physically moves):** `dolly` (in/out along the lens axis), `truck` (left/right laterally), `pedestal` (up/down vertically) +- **Rotation (rig stays put, camera rotates):** `pan` (yaw, left/right), `tilt` (pitch, up/down), `roll` (Dutch / Z-axis) +- **Lens-only (rig and body don't move):** `zoom` (focal length change), `rack focus` / `pull focus` / `focus tracking` (focal-plane change) + +dolly ≠ zoom; pan ≠ truck. VEO follows whichever token leads. + ## VEO Lens Effects (Unique) VEO specifically responds to optical effects most models ignore: | Effect | Prompt Language | |--------|----------------| -| **Rack focus** | "rack focus from foreground flower to background figure" | +| **Rack focus** | "rack focus from foreground flower to background figure" (snap shift) | +| **Pull focus** | "slow pull focus from the candle in the foreground to the doorway behind" (gradual, slower than rack) | +| **Focus tracking** | "focus tracks the runner as she crosses frame; background stays soft" (focus follows a moving subject) | | **Dolly zoom (vertigo)** | "vertigo effect as character realizes the truth" | | **Fisheye** | "fisheye lens distortion, skatepark POV" | -| **Lens flare** | "anamorphic lens flare from setting sun" | +| **Anamorphic lens flare** | "anamorphic lens flare streaking horizontally from setting sun" | + +These three focus modes (rack, pull, tracking) are different — VEO 3.1 honors the distinction per the paper. ## VEO Art Movement References diff --git a/skills/creative/storytelling.md b/skills/creative/storytelling.md index fbaa77d0..19094e5e 100644 --- a/skills/creative/storytelling.md +++ b/skills/creative/storytelling.md @@ -57,6 +57,32 @@ For a **3-minute explainer video** (scale proportionally for other lengths): | 3 min | 3-5 | 8s | 22s | 100s | 30s | 15s | | 5 min | 5-8 | 10s | 30s | 180s | 50s | 20s | +## Anti-Subjective Rule + +> Hooks, beats, and section descriptions in OpenMontage scripts must describe the **visual cause** of the emotion, not the emotion itself. The CMU/Harvard CHAI study showed that subjective phrasing varies wildly across annotators and across model interpretations — so it does not constrain pixels and it doesn't reliably guide downstream generation tools. +> +> | Avoid | Use instead | +> |---|---| +> | "epic reveal" | "wide aerial pull-back; subject silhouetted against rising sun" | +> | "inspiring moment" | "low angle on the subject's face; light catches the edge of a tear" | +> | "moody atmosphere" | "low-key key light, lifted shadows by 2 stops, fog volumetrics" | +> | "powerful music swell" | "music drops out at 0:42, holds 1.5s of silence, returns with low taiko at half tempo" | +> +> The rule applies to script narration AND to the metadata fields scene-director consumes. For the universal vocabulary that names these visual primitives, see `skills/creative/video-gen-prompting.md`. + +## Subject Transitions in the Script + +When a script beat introduces a new subject, kills one off, or hands focus from one subject to another, **name the transition explicitly** so the scene-director doesn't have to infer it. The CMU/Harvard taxonomy uses four labels: + +| Label | What it means | +|---|---| +| **revealing** | A new subject enters frame or is uncovered (door opens, camera pans to find them, fog clears). | +| **disappearing** | An existing subject leaves frame or is removed (walks out, fades, eclipsed). | +| **switching** | Focus jumps from subject A to subject B (cut, rack focus, camera whip). | +| **complex-alternating** | Multiple subjects trade focus repeatedly within a beat (debate cross-cutting, ensemble action). | + +Add 1-2 sentences in the beat describing the mechanism (cut, pan, reveal-by-light, etc.). This propagates into the scene_plan as a transition primitive. + ## Hook Types | Type | Pattern | Best For | @@ -114,6 +140,20 @@ Don't explain the answer. **Reconstruct the reasoning path** so the viewer feels **Progressive Revelation:** Never show the full picture at once. Build visuals layer by layer. Each layer arrives exactly when the narration references it. +## Camera Intent Per Beat + +When writing a beat, attach one line of camera intent so the scene-director doesn't have to invent it from a blank slate. Use the universal vocabulary in `skills/creative/video-gen-prompting.md` (Subject / Subject Motion / Scene / Spatial Framing / Camera). One line is enough — the scene-director will expand it. + +Example beat: + +``` +[0:30] Concept 1 — atoms aren't tiny planets +Narration: "We grew up imagining electrons as tiny planets orbiting the nucleus..." +Camera intent: medium shot of stylized atom; slow rotation; deep focus. +``` + +The camera-intent line is consumed verbatim by the scene-director's 5-aspect spec — keep it concrete, no mood adjectives. + ## Pacing Rules | Rule | Value | Source | diff --git a/skills/creative/video-gen-prompting.md b/skills/creative/video-gen-prompting.md index 1b606be1..95628146 100644 --- a/skills/creative/video-gen-prompting.md +++ b/skills/creative/video-gen-prompting.md @@ -24,18 +24,50 @@ For model-specific tips, see the linked guides below. | **Kling 2.6** | [Kling Prompt Guide](https://fal.ai/learn/devs/kling-2-6-pro-prompt-guide) | 4-part structure. Supports `++emphasis++` syntax for key elements. | | **Wan 2.1 / CogVideoX** | Use this generic guide | No official prompt guide. Standard cinematographic vocabulary works well. | +## Order Matters + +When listing multiple subjects or events: + +- **Temporal order** when events unfold over time ("First X enters, then Y reacts"). +- **Prominence order** when temporal isn't relevant — humans before objects, largest/most-centered first, then secondary subjects. + +## Self-Contained Prompt + +> Write the prompt so that someone who has never seen the intended video could picture the subjects, scene, motion, and camera work from your text alone. If a reader could not picture it, a generation model will not render it. + ## Universal Prompt Formula -All video generation models respond to this structure. Include what's relevant, omit what's not. +Prior work (CMU/Harvard, "Building a Precise Video Language with Human-AI Oversight") shows VLMs reliably describe subject + scene but fail on motion, spatial, and camera. **Forcing prompts to fill all five slots is the highest-leverage change.** + +The OpenMontage canonical 5-aspect skeleton: ``` -[Shot type/framing] + [Camera movement] + [Subject description] + -[Action/motion in beats] + [Setting/environment] + [Lighting] + -[Style/aesthetic] + [Audio/atmosphere] +[Subject] type + key visual attributes + how to disambiguate when multiple +[Subject Motion] actions in temporal order; subject↔object and subject↔subject interactions; group action +[Scene] overlays (separately!) + POV + setting + time of day + scene dynamics +[Spatial] shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative + — and how those CHANGE during the clip +[Camera] playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement ``` **Shorter prompts = more creative freedom. Longer prompts = more control.** +### Prompt Length by Model + +Empirical sweet spots from the paper's Section 6 findings — different models reward different prompt densities: + +| Model | Sweet Spot | Notes | +|---|---|---| +| Seedance 2.0 | 200–400 words for hero shots, 80–150 for inserts | Reward long, structured 5-aspect prompts | +| Wan 2.2 | 200–400 words | Fine-tuned on long captions | +| Sora 2 / VEO 3.1 | 100–250 words | Plateau past ~250 | +| LTX-2 | ≤ 80 words | Degrades past that, keep tight | +| Runway Gen-4 | ≤ 60 words | "Focus on motion, not appearance" | + +### Overlays Are Not Scene Depth + +> Overlays (titles, HUD, subtitles, watermarks, framing graphics) are NOT part of the scene's foreground/midground/background depth axis. List them separately with content and placement. Never say "overlay in the foreground." + --- ## Camera Shot Types @@ -58,23 +90,56 @@ All video generation models respond to this structure. Include what's relevant, ## Camera Movements -| Movement | What It Does | Best For | -|----------|-------------|----------| -| **Static / fixed** | No movement | Dialogue, contemplation, stability | -| **Pan** (left/right) | Rotates horizontally | Revealing a scene, following action | -| **Tilt** (up/down) | Rotates vertically | Revealing height, slow reveal | -| **Dolly in / out** | Physically moves toward/away | Building tension, emphasis | -| **Truck** (left/right) | Moves sideways | Parallels subject movement | -| **Pedestal** (up/down) | Moves vertically | Smooth elevation changes | -| **Crane shot** | Sweeping vertical arcs | Epic reveals, transitions | -| **Tracking / follow** | Follows subject | Action sequences, walk-and-talk | -| **Arc shot** | Circles around subject | Dramatic emphasis, 360° reveal | -| **Zoom** (in/out) | Lens focal length change | Quick emphasis (cheaper than dolly) | -| **Whip pan** | Extremely fast pan (blurs) | Transitions, energy, surprise | -| **Handheld / shaky cam** | Unstable, human feel | Documentary, urgency, realism | -| **Aerial / drone** | High altitude, smooth | Landscapes, establishing shots | -| **Slow push-in** | Gradual forward movement | Building intimacy or tension | -| **Dolly zoom (vertigo)** | Dolly one way, zoom opposite | Disorientation, revelation | +The paper shows current models confuse translation, rotation, and lens-only changes — group your prompts so the model can't conflate them: + +| Group | Primitives | Rule | +|---|---|---| +| **Translation** (camera physically moves) | dolly in/out, truck left/right, pedestal up/down | "dolly forward toward subject" | +| **Rotation** (camera pivots in place) | pan left/right, tilt up/down, roll CW/CCW | "pan right across the room" | +| **Lens-only** (no camera move) | zoom in/out, rack focus, pull focus, focus tracking | "zoom in" ≠ "dolly in" | +| **Hybrid / signature** | dolly zoom (vertigo), arc/orbit, crane, whip pan, tracking/follow, handheld | "vertigo" only at moments of revelation | +| **Stillness states** | static (NO movement at all — strict), micro-shake, locked-off | "static" requires zero movement, focus change, or zoom | + +> **dolly ≠ zoom.** dolly is camera translation; zoom is focal-length change. Models follow whichever token dominates. **pan ≠ truck.** pan rotates, truck translates laterally. + +> **Static shot is strict.** A static shot has zero movement, zero focus change, zero zoom. If any of those occur, do NOT write "static camera" — pick the right movement primitive. + +## Camera Height (relative to ground) + +| Primitive | Example | +|---|---| +| Aerial-level | "drone-altitude wide of the city" | +| Overhead-level | "rooftop height looking across the street" | +| Eye-level | "framed at eye level" | +| Hip-level | "hip-height tracking shot" | +| Ground-level | "low to the ground, ankle height" | +| Water-level | "skimming the water surface" | +| Underwater | "submerged below the surface" | + +## Camera Angle (relative to subject) + +| Primitive | Definition | +|---|---| +| **Bird's-eye** | strict top-down. Not the same as aerial. | +| High angle | looking down on subject | +| Level angle | camera and subject at same height | +| Low angle | looking up at subject | +| Worm's-eye | looking straight up | +| **Dutch angle (fixed)** | tilted horizon held steady | +| **Dutch angle (rolling)** | horizon tilt changes during shot | + +> **bird's-eye = strict top-down. aerial = altitude.** A drone shot at 45° looking down is a high angle from aerial height, NOT bird's-eye. + +## Point of View (POV) + +| POV | Example | +|---|---| +| First-person | "the camera follows the character's viewpoint as they walk" | +| Drone | "aerial drone footage of city skyline" | +| Over-the-shoulder | "OTS framing of the laptop screen" | +| Top-down oblique | "top-down view of the chess board, tilted slightly" | +| Dashcam | "vehicle dashcam framing of the road" | +| Objective / Neutral | (default — use when no specific POV) | ## Lighting Vocabulary @@ -100,14 +165,51 @@ All video generation models respond to this structure. Include what's relevant, | Effect | Result | |--------|--------| -| **Shallow depth of field** | Subject sharp, background bokeh | -| **Deep focus** | Everything sharp, foreground to background | | **Wide-angle lens** (24-35mm) | Broader view, exaggerated perspective | | **Telephoto** (85mm+) | Compressed perspective, subject isolation | | **Anamorphic** | Stretched aspect, signature lens flares | | **Lens flare** | Streaks from bright light hitting lens | -| **Rack focus** | Shift focus between subjects in-shot | -| **Fisheye** | Ultra-wide, barrel distortion | + +### Lens Distortion + +The paper distinguishes two primitives that models honor as separate effects — they are NOT interchangeable: + +| Primitive | Effect | +|---|---| +| **Fisheye** | extreme curvature, edges bent strongly outward | +| **Barrel** | mild distortion, straight lines bow slightly outward | + +### Focus / Depth of Field + +| Primitive | Definition | +|---|---| +| Deep focus | everything sharp, FG to BG | +| Shallow DoF | subject sharp, background bokeh | +| Extremely shallow DoF | razor-thin focal plane | +| Rack focus | shifts focus between two subjects mid-shot | +| Pull focus | gradual focus shift (slower than rack) | +| Focus tracking | focus follows a moving subject | + +When DoF changes during a shot, label start AND end focal plane (FG/MG/BG/out-of-focus). + +## Subject Transitions + +When subjects enter, leave, or hand off focus, name the transition explicitly: + +| Primitive | When | +|---|---| +| **Subject revealing** | a new subject enters frame (by subject movement OR camera movement) | +| **Subject disappearing** | a subject exits frame | +| **Subject switching** | focus shifts from one subject to another (often via rack focus or camera move) | +| **Complex alternating** | subjects alternate focus multiple times | + +Always name the cause: "by subject movement" or "by camera movement". This unlocks reveal-style camerawork in multi-shot prompts. + +## Identity Anchoring for Multi-Shot Prompts + +> Models lose character identity across cuts unless you re-state it. In every shot of a multi-shot prompt, repeat the same 3–6 disambiguating visual attributes for each named subject verbatim. Pronouns and "the same character" do not work. +> +> Example: "Aang — bald, blue arrow tattoo on forehead, orange-and-yellow robes — plants his staff. … Aang — bald, blue arrow tattoo on forehead, orange-and-yellow robes — turns to camera." ## Style & Aesthetic References @@ -137,10 +239,23 @@ All video generation models respond to this structure. Include what's relevant, ## Temporal Effects +### Playback Speed + +The paper defines six explicit playback-speed primitives. Use the right one — they're not synonymous: + +| Primitive | Definition | +|---|---| +| Time-lapse | events significantly faster than real time (clouds racing) | +| Fast-motion | slightly faster than real (1x–3x) | +| Slow-motion | slower than real | +| Stop-motion | frame-by-frame discrete movements | +| Speed-ramp | mix of fast and slow within the same shot | +| Time-reversed | plays in reverse | + +### Other Temporal Devices + | Effect | Use | |--------|-----| -| **Slow motion** | Emphasis, beauty, impact | -| **Time-lapse** | Passage of time, processes | | **Freeze-frame** | Dramatic pause | | **Rapid cuts** | Energy, urgency | | **Continuous / long take** | Immersion, tension | @@ -160,6 +275,15 @@ Put dialogue in quotation marks: `Character says: "Hello world."` ## What to Avoid +> **Replace emotional adjectives with the visual cause of the emotion.** +> - "sad character" → "tears on cheek, shoulders slumped, staring at empty chair" +> - "cinematic mood" → "low-key Rembrandt key + 35mm anamorphic + crushed shadows, lifted-by-2-stops shadow detail" +> - "epic" → "low-angle, 24mm wide, sun directly behind subject, lens flare on the rim" +> +> "Inspiring," "powerful," "moody," "epic" do not constrain pixels. + +> **Static shot is strict.** A static shot has zero movement, zero focus change, zero zoom. If any of those occur, do NOT write "static camera" — pick the right movement primitive. + | Don't | Why | Do Instead | |-------|-----|-----------| | "Beautiful scene" | Too vague, no visual info | "Wet cobblestone street, warm streetlamp glow reflecting in puddles" | diff --git a/skills/meta/reviewer.md b/skills/meta/reviewer.md index 43f6235e..b440ca8c 100644 --- a/skills/meta/reviewer.md +++ b/skills/meta/reviewer.md @@ -6,6 +6,18 @@ After completing any pipeline stage's work — before checkpointing. You are the Every stage gets reviewed. No exceptions. The review quality determines whether the final video is worth watching. +## Critique Quality (CHAI Rules) + +> Findings ≠ critiques. A finding identifies a problem; a critique tells the next stage how to fix it. The CMU/Harvard CHAI study ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2) showed that critique quality, measured on three axes, directly governs downstream output quality. Apply all three to every reviewer pass. +> +> **Accurate.** Every finding must reference a concrete artifact field, line number, or visible asset frame. Forbid hallucinated criticism — if you cannot point to where the problem is, you are guessing. +> +> **Complete.** A reviewer pass that catches one mistake while missing a second is worse than scoring "needs another pass" and continuing. If you find one critical issue, scan for the rest of the same class before returning. Pattern-match: where else in this artifact could the same mistake be hiding? +> +> **Constructive.** Every "critical" finding MUST propose a concrete fix, not just identify the problem. "Caption is wrong" → "Caption says 'man on the right'; the man is on the left of the frame. Replace with 'the man on the left of the frame.'" If you cannot propose a fix, label the finding as "investigation" not "critical." +> +> Removing any of these three properties measurably hurts pipeline output. The reviewer is the choke point — be rigorous. + ## Protocol ### Step 1: Load Review Context @@ -27,9 +39,10 @@ First, the non-negotiable check: For each `review_focus` item from the manifest: 1. Evaluate the artifact against this specific criterion 2. Assign a severity: - - **critical** — Must fix before proceeding. The artifact is broken, incomplete, or dangerously wrong. - - **suggestion** — Should fix. Improves quality significantly but doesn't block progress. - - **nitpick** — Could fix. Minor polish that's nice-to-have. + - **critical** — Must fix before proceeding. The artifact is broken, incomplete, or dangerously wrong. **Per CHAI rules, every critical finding MUST carry a `proposed_fix` (concrete replacement text, exact field value, or specific corrective action). A critical finding without a proposed fix is downgraded to `investigation`.** + - **suggestion** — Should fix. Improves quality significantly but doesn't block progress. **Suggestions MUST carry a `proposed_change` describing how to improve.** + - **nitpick** — Could fix. Minor polish that's nice-to-have. May stand alone without a proposed change. + - **investigation** — A real concern but you cannot pinpoint the fix. Surface it for the next round; do not block on it. 3. Write a specific, actionable finding (not vague) **Good finding:** "Section 3 narration is 180 words for a 10-second window — that's 1080 wpm, impossible to speak. Cut to 25 words." diff --git a/skills/meta/video-reference-analyst.md b/skills/meta/video-reference-analyst.md index 4ad37294..35232c65 100644 --- a/skills/meta/video-reference-analyst.md +++ b/skills/meta/video-reference-analyst.md @@ -38,7 +38,8 @@ video_analyzer.execute({ ``` Read the resulting VideoAnalysisBrief. Before proceeding, present a summary to the -user. This is NOT a raw dump. It's a conversational interpretation: +user. This is NOT a raw dump. It's a conversational interpretation, and it MUST be +structured by the 5 aspects so downstream stages can lift fields directly: ``` "I've watched the video. Here's what I see: @@ -48,12 +49,22 @@ user. This is NOT a raw dump. It's a conversational interpretation: **Structure:** [X scenes over Y seconds, pacing style] **Motion:** [N of M scenes are motion clips / animated stills / static images. This video uses [AI-generated video clips / still images with pan-zoom / a mix].] + +**5-aspect breakdown (per shot or per shot-group):** +- Subject: [type, count, attributes; subject transitions across shots: revealing / disappearing / switching / complex-alternating; or N/A] +- Subject Motion: [actions in temporal order; interactions; or N/A] +- Scene: [overlays (text/graphics) listed separately; POV (drone/OTS/macro/etc.); setting; time of day; dynamics] +- Spatial Framing: [shot size; subject position; depth; height-relative; how it changes] +- Camera: [playback speed; lens; height; angle; focus/DoF; steadiness; movement] + **What makes it work:** [2-3 specific things — the hook technique, the pacing, the visual transitions, the narration style] Now let me check what I can do with your current setup..." ``` +The 5-aspect block above is the **canonical form** that `proposal-director`, `script-director`, and `scene-director` will read. Do not collapse it back into prose — keep the labels. + **Motion classification is critical.** The VideoAnalysisBrief now includes per-scene `motion_type` ("motion_clip", "animated_still", "static_image") and `flow_variance`. Use this to determine the production approach: @@ -81,6 +92,22 @@ Update the brief's `content_analysis`, `style_profile`, and `replication_guidanc fields with your visual observations. This is where the analysis becomes truly comprehensive — the tools provide structure; your vision provides understanding. +### 5-Aspect Structured Output (MANDATORY) + +The analyst's report MUST break down the reference video into the **five aspects** from the CMU/Harvard CHAI study (also the canonical structure used in `skills/creative/video-gen-prompting.md`). A narrative-only summary is no longer sufficient — downstream stages (proposal, script, scene-director) ingest the 5-aspect form directly without re-parsing prose. + +**Decision-tree captioning policy.** For each detected shot, walk all five aspects in order: + +> - **Subject:** type, attributes (count, age, role, costume, distinguishing features), multiple-subject disambiguation, transitions across shots (revealing / disappearing / switching / complex-alternating). +> - **Subject Motion:** actions in temporal order; group/interaction patterns (parallel, sequential, reactive); locomotion vs gesture vs facial. +> - **Scene:** **overlays separately** (text, lower thirds, graphics, watermark — call these out as their own layer, do not merge into setting) + POV (drone, aerial, OTS, macro, top-down, dashcam, FPV, handheld, locked-off) + setting + time of day + dynamics (weather, particles, crowd movement). +> - **Spatial Framing:** shot size (ECU/CU/MS/WS/EWS), subject position in frame, depth (foreground/midground/background usage), height-relative (above/at/below subject) — and how each of these **changes** across the shot if the camera or subject moves. +> - **Camera:** playback speed (real-time / slow-mo / time-lapse), lens distortion (anamorphic, fish-eye, tilt-shift), height (ground / eye / overhead), angle (high / low / Dutch), focus / DoF (rack focus, deep focus, shallow), steadiness (locked / handheld / gimbal), movement (push / pull / pan / tilt / dolly / truck / crane / orbit). +> +> **Mark any aspect explicitly as N/A** if it doesn't apply (e.g., "Subject: N/A — pure scenery shot," or "Scene overlays: N/A — no graphics"). **Silent omission is the most common analyst failure** and produces ambiguous downstream prompts. + +See `skills/creative/video-gen-prompting.md` for primitive definitions and the canonical vocabulary used at every aspect. + ### Step 2: Capability Audit Run standard preflight: diff --git a/skills/pipelines/animation/scene-director.md b/skills/pipelines/animation/scene-director.md index 858ad8ac..f5a247fe 100644 --- a/skills/pipelines/animation/scene-director.md +++ b/skills/pipelines/animation/scene-director.md @@ -85,9 +85,25 @@ Recommended metadata keys: - `tool_path_map` - `reusable_motifs` -### 5. Quality Gate +### 5. 5-Aspect Scene-Plan Checklist + +> Every scene must specify all five aspects, BUT the load shifts with the scene's `animation_mode`. Manim and other diagrammatic/programmatic scenes care most about **Subject** and **Spatial Framing** — Camera and Subject Motion in the cinematographic sense often map to N/A or to abstract equivalents. AI-video / `image_animation` / `anime_scene` scenes care about all five and behave like cinematic shots. Marking an aspect as N/A is allowed but must be explicit per scene; silent omission is forbidden. +> +> 1. **Subject** — type + key visual attributes; for Manim, the equation/object/graph being foregrounded; for `anime_scene`, the character or environment in focus. +> 2. **Subject Motion** — for Manim, the order of `Create`/`Transform`/`FadeIn` and what each animation conveys; for AI-video, the actions and interactions in temporal order. +> 3. **Scene** — overlays (separately!) + POV + setting + time of day + scene dynamics. For Manim, "setting" is the canvas background + axis style; for `anime_scene`, the environment + lighting gradient. +> 4. **Spatial Framing** — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE. Manim cares about layout grid + element positions; AI-video cares about full cinematographic framing. +> 5. **Camera** — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement. For Manim and pure motion-graphics, default to N/A unless using a virtual camera move (`MoveCamera`, `self.frame`). For `anime_scene` and AI-video, specify fully. +> +> Tie this back to `animation_mode` in scene metadata: a Manim scene that lists Camera fully is over-specified; an AI-video scene that omits Camera is under-specified. See `skills/creative/video-gen-prompting.md` for the primitive vocabulary. + +> **Overlays callout.** Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds, `hero_title`, `section_title`, `provider_chip`) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (`overlays: [...]`) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output. + +### 6. Quality Gate - every scene has a clear timing intent, +- the 5-aspect checklist is satisfied for the scene's `animation_mode` (with explicit N/A where appropriate), +- overlays live under `overlays:`, never inside the framing description, - the transition system is limited and meaningful, - the tool path is explicit, - the sequence feels like one designed system. diff --git a/skills/pipelines/cinematic/asset-director.md b/skills/pipelines/cinematic/asset-director.md index 0a818561..73f45a26 100644 --- a/skills/pipelines/cinematic/asset-director.md +++ b/skills/pipelines/cinematic/asset-director.md @@ -101,6 +101,22 @@ Recommended metadata keys: - `generated_support_assets` - `rights_notes` +### Pre/Post Self-Review for Generation Prompts + +> Before sending a prompt to any image or video generation tool, run a three-step self-review modeled on the CHAI oversight loop ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2). Cost is small (no extra tool calls); benefit is large (avoids wasted generations). For cinematic, this matters most for **hero-frame prompts** — one bad hero frame ruins the piece, and hero frames are the most expensive shots to regenerate. +> +> **Step 1 — Pre-caption pass.** Write the prompt the way you'd write it today. Do not over-edit; aim for a complete first draft. +> +> **Step 2 — Critique pass.** Score the draft against the 5-aspect checklist (Subject / Subject Motion / Scene / Spatial Framing / Camera). For each aspect: +> - Is it specified? If not, is the omission deliberate (e.g., "no subject — scenery shot") or accidental? +> - Are confusable terms disambiguated? (dolly vs zoom, pan vs truck, bird's-eye vs aerial, fisheye vs barrel, full shot vs close-up) +> - Are emotional adjectives ("epic", "moody", "cinematic") replaced with their visual causes (low-key lighting, slow push-in, anamorphic flare, deep shadows)? +> - For multi-shot prompts and identity-anchored hero frames: is identity anchored verbatim across shots? +> +> **Step 3 — Post-caption pass.** Rewrite filling the missing aspects, fixing confusable terms, and replacing subjective language. The post-caption is what gets sent to the generation tool. +> +> Log the (pre, critique, post) triplet in the asset metadata for traceability. This mirrors the CHAI workflow and creates a record the reviewer can audit. + ### 5. Quality Gate - source and support assets are clearly distinguished, diff --git a/skills/pipelines/cinematic/scene-director.md b/skills/pipelines/cinematic/scene-director.md index 348baf24..d03817bb 100644 --- a/skills/pipelines/cinematic/scene-director.md +++ b/skills/pipelines/cinematic/scene-director.md @@ -49,11 +49,26 @@ Recommended metadata keys: - `title_card_rules` - `support_insert_rules` -### 5. Quality Gate +### 5. 5-Aspect Scene-Plan Checklist + +> Every scene beat — and especially every hero frame — must specify all five aspects. Cinematic relies on a small number of memorable frames; vague hero-frame specs are the single most common failure mode and produce unpredictable model output. Marking an aspect as N/A is allowed but must be explicit (e.g., "no subject — establishing scenery shot"). Silent omission is forbidden. +> +> 1. **Subject** — type + key visual attributes; if multiple, how to disambiguate. For hero frames, identity must be anchored verbatim across shots. +> 2. **Subject Motion** — actions in temporal order; subject↔object / subject↔subject interactions. +> 3. **Scene** — overlays (separately!) + POV + setting + time of day + scene dynamics. +> 4. **Spatial Framing** — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE across the beat. +> 5. **Camera** — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement. +> +> See `skills/creative/video-gen-prompting.md` for the primitive vocabulary. + +> **Overlays callout.** Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds, name plates, end-tag cards) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (`overlays: [...]`) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output. + +### 6. Quality Gate - every beat has a scene treatment, -- hero frames are identifiable, +- hero frames are identifiable AND fully specified across all 5 aspects, - support inserts are justified, +- overlays are recorded under `overlays:`, never inside the depth/framing description, - the visual language stays consistent across the piece. ## Common Pitfalls diff --git a/skills/pipelines/explainer/asset-director.md b/skills/pipelines/explainer/asset-director.md index e521533f..ac9f091a 100644 --- a/skills/pipelines/explainer/asset-director.md +++ b/skills/pipelines/explainer/asset-director.md @@ -178,6 +178,22 @@ Assemble all generated assets into the manifest: } ``` +### Pre/Post Self-Review for Generation Prompts + +> Before sending a prompt to any generation tool — `image_selector`, `diagram_gen`, `video_selector`, even `code_snippet` styling prompts — run a three-step self-review modeled on the CHAI oversight loop ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2). Cost is small (no extra tool calls); benefit is large (avoids wasted generations). This applies just as strongly to `diagram_gen` Mermaid prompts and `image_selector` illustration prompts as it does to video generation — bad explainer visuals fail in the same way: missing subject, missing framing, vague "make it look educational." +> +> **Step 1 — Pre-caption pass.** Write the prompt the way you'd write it today. Do not over-edit; aim for a complete first draft. +> +> **Step 2 — Critique pass.** Score the draft against the 5-aspect checklist (Subject / Subject Motion / Scene / Spatial Framing / Camera). For each aspect: +> - Is it specified? If not, is the omission deliberate (e.g., "Camera N/A — Remotion native scene", "no subject motion — static diagram") or accidental? +> - Are confusable terms disambiguated? (dolly vs zoom, pan vs truck, bird's-eye vs aerial, fisheye vs barrel, full shot vs close-up; for diagrams: flowchart vs sequence vs state diagram, top-down vs left-right) +> - Are emotional adjectives ("clean", "professional", "modern") replaced with their visual causes (sans-serif typography, generous whitespace, monochromatic palette with one accent)? +> - For multi-shot prompts: is identity anchored verbatim across shots? For `image_selector` prompts that recur (a character or world appearing in multiple scenes), are consistency anchors specified verbatim? +> +> **Step 3 — Post-caption pass.** Rewrite filling the missing aspects, fixing confusable terms, and replacing subjective language. The post-caption is what gets sent to the generation tool. +> +> Log the (pre, critique, post) triplet in the asset metadata for traceability. This mirrors the CHAI workflow and creates a record the reviewer can audit. + ### Step 7: Verify All Assets **Existence check:** diff --git a/skills/pipelines/explainer/scene-director.md b/skills/pipelines/explainer/scene-director.md index 86787aa8..0d37827a 100644 --- a/skills/pipelines/explainer/scene-director.md +++ b/skills/pipelines/explainer/scene-director.md @@ -156,6 +156,20 @@ If the video includes narration, the script **must** be written to fit the video - If narration exceeds video by >1s, either trim the script and regenerate, or extend the video's closing scene. - Always run `composition_validator` before rendering to catch mismatches automatically. +### Step 4c: 5-Aspect Scene-Plan Checklist + +> Every scene must specify all five aspects. For diagram, chart, and Remotion-native scenes, "Subject" can map to a foregrounded data element and "Camera" can be marked N/A — but only EXPLICITLY (e.g., `"camera": "N/A — Remotion native scene, no virtual camera"`). Silent omission is the most common failure mode and produces unpredictable model output, brittle prompts, and reviewer churn. +> +> 1. **Subject** — type + key visual attributes; if multiple, how to disambiguate. For diagram/chart scenes, this is the foregrounded data element (the node, the bar, the KPI being highlighted). For generated images, it's the person/object/concept being illustrated. +> 2. **Subject Motion** — actions in temporal order; for animated diagrams, the order in which nodes/edges/values appear or change. +> 3. **Scene** — overlays (separately!) + POV + setting + time of day + scene dynamics. For Remotion scenes, "setting" maps to background treatment + theme. +> 4. **Spatial Framing** — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE. For static Remotion scenes, document the layout grid + which element occupies the visual center. +> 5. **Camera** — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement. Mark N/A for native-Remotion scenes; specify fully for `generated`/`broll`/`image_animation` scenes. +> +> See `skills/creative/video-gen-prompting.md` for the primitive vocabulary. + +> **Overlays callout.** Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds, section_title bars, stat_reveal chips, hero_title overlays, provider chips) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (`overlays: [...]`) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output. + ### Step 5: Validate Against Playbook The style playbook constrains your visual choices: