- New: Higgsfield video provider with multi-model routing (Kling 3.0, Veo 3.1, Sora 2, WAN 2.5, Soul Cinema) and Soul ID character consistency
- Updated: Runway provider with gen4_aleph and gen3a_turbo models, proper pixel-ratio mapping, probe_output, watermark param, RUNWAYML_API_SECRET env var support
- Docs: Updated provider counts (12→13), tool counts (51→52), added Higgsfield setup/pricing sections across README, AGENT_GUIDE, ARCHITECTURE, and PROVIDERS
New capture layer for the screen-demo pipeline: FFmpeg for quick CLI-driven
recording, Cap integration for polished recordings with webcam overlay and
cursor effects. Selector presents both options and routes based on availability.
Route Chirp and Journey voice families to the v1beta1 API endpoint
automatically. Adds beta voice detection, dynamic API version
selection, and Chirp3-HD cost estimation ($30/1M chars).
Compose-director had contradictory instructions: Step 2 described Remotion
captions/audio, but Steps 5/5b gave detailed FFmpeg code that agents followed
instead. This caused three failures in production: FFmpeg subtitles instead of
Remotion CaptionOverlay, missing audio (mixed externally but never embedded in
Remotion props), and skipped audio verification in post-render review.
Changes:
- compose-director: Remotion is now DEFAULT for audio, captions, text overlays;
FFmpeg is labeled FALLBACK only. Post-render review has mandatory ffprobe gate
and audio transcription with explicit stop conditions.
- remotion.md: routing table updated (captions/audio → Remotion), added universal
Post-Render Verification Protocol for all pipelines (only 2/10 had one).
- scene-director, asset-director: added pitfall for AI-generated text in CTA
screens — must use Remotion text_card for verbatim text.
- image-provider-usage: added Recraft V4 caveat (style param causes 422 on fal.ai).
- recraft_image.py: documented the style parameter 422 issue inline.
- AGENT_GUIDE.md: add "never read source code" rule — skills are the
interface, not .py files
- animation.yaml: add Layer 2 skill-first guardrail in assets stage
- video-reference-analyst.md: Step 4b now mandates Layer 2 before Layer 3,
explicitly forbids reading implementation code
- CinematicRenderer: add TikTok-style CaptionOverlay and separate music
track support (narration + music as independent audio layers)
- cinematic/types.ts: add CinematicCaptionConfig and music prop types
Chirp3-HD is near-free ($0.003 vs $0.30+ per script), expressive,
24kHz, and available without quota limits. ElevenLabs demoted to
voice-cloning-only recommendation.
New Step 4b requires the agent to read agent_skills from every tool
before writing generation prompts. Discovered during UAT: agent
generated video clips, images, and TTS without reading provider-specific
prompting guidance, violating AGENT_GUIDE governance.
Proposals must now specify clip duration strategy (prefer 10s over 5s
to halve API costs) and list Layer 3 skills that must be read before
generating assets. Both were governance gaps found during UAT.
The instructions parameter was passed as None when not provided,
causing a 400 error. Now only included when explicitly set. Also
adds speed passthrough support.
test_01_tts.py (ElevenLabs), test_02_image_gen.py (DALL-E + FLUX),
and test_03_music.py (ElevenLabs Music) were burning ~$0.44 per run
against live API keys. These were run repeatedly and exhausted the
ElevenLabs starter plan quota (30K characters).
Agent must run tts_selector preflight to check available providers
instead of defaulting to ElevenLabs. Supports ElevenLabs, Google TTS,
OpenAI TTS, and Piper (offline).
The agent must lock the audio approach (single narrator vs. character
dialogue vs. both) during Step 3, not defer it to script/compose stage.
Proposals now include voice casting with specific voice IDs.
Remotion is now the default composition engine for ALL final renders
when available — video clips, images, mixed content. FFmpeg is only
used as fallback when Remotion is not installed or for standalone
operations (trim, transcode). Also fixes three FFmpeg fallback bugs:
profile + copy codec conflict, stream order mapping, and segment
seeking for audio-first containers.
The reference analyst skill went straight from capability audit to
creative proposals with no research. This caused the agent to propose
concepts based solely on the reference analysis and its own knowledge,
missing content landscape context, technique best practices, and
subject-matter depth.
Added Step 3b between critical questions and creative proposals:
- Content landscape scan (3-5 similar existing videos)
- Style/technique research (AI model strengths, prompting patterns)
- Subject-matter research (facts, tropes, hooks)
- 2-3 minute time budget — lightweight, not full research-director
Four gaps found during user acceptance testing of the reference-video
production workflow:
- video_selector: expose aspect_ratio, duration, reference_image_path,
reference_image_url, and image_url in schema so agents can discover
these critical params. Auto-upload local images when the selected
provider requires a URL.
- _shared.py: add upload_image_fal() for local→URL image bridging via
fal.ai storage. Fix upload_image_heygen() to try v2 presigned upload
before falling back to fal.ai (old /v1/asset endpoint returns 404).
- kling_video: call probe_output() so response includes output_path,
duration_seconds, file_size_mb, video dimensions, and codec info.
The AGENT_GUIDE establishes Remotion as the preferred composition engine
over FFmpeg, but the reference analyst skill was presenting them as peer
options. This caused the agent to default to FFmpeg during proposals.
- Capability audit template now labels Remotion as "preferred" and FFmpeg
as "fallback only"
- Added explicit composition engine priority note
- Proposal template separates Composition and Motion as distinct lines
Video analyzer now uses Farneback dense optical flow to classify each
scene as motion_clip, animated_still, or static_image. This lets the
agent correctly identify whether a reference video uses AI-generated
video clips vs still images with pan/zoom — and plan the right pipeline.
Changes:
- video_analyzer.py: new Step 3b with _classify_scene_motion() and
_read_frame_at() helpers; updated _needs_motion() to use per-scene
motion data instead of pacing heuristic alone
- video-reference-analyst.md: added Motion line to summary template
and instructions to read motion_type field before proposing tools
Entry point files (CLAUDE.md, AGENTS.md, .windsurfrules, .cursorrules,
copilot-instructions.md) previously contained useful snippets that let
agents skip reading the full AGENT_GUIDE. Now they contain zero usable
instructions — only a mandatory redirect — so any assistant is forced
to read AGENT_GUIDE.md before acting on user requests.
Implements the 2026-04-02 transformation spec (Phases 1-8) and fixes all
critical bugs found during 5-pipeline E2E testing.
Governance & Decision Intelligence:
- Pipeline-specific stage order in checkpoint (replaces global STAGES list)
- Provider scoring engine (lib/scoring.py) with 7-dimension weighted ranking
- Decision log artifact enforced at proposal/idea stage across all 10 pipelines
- Delivery promise classifier prevents silent motion-to-still downgrades
- Structured shot language in scene_plan schema (camera, lens, lighting, DOF)
- Variation checker and slideshow risk scorer block samey output before render
- Creative intake, capability extension, and creative-intake meta skills
- Final self-review artifact with 5 mandatory checks before presenting output
- Source media review contract for user-supplied footage
Render & Theme System:
- Remotion AnimatedBackground now derives colors from playbook (no more hardcoded
dark blue fintech gradient on every video)
- video_compose builds custom ThemeConfig from playbook YAML colors/fonts —
custom playbooks flow through to Remotion automatically
- Explainer component wires theme to all child components (charts, cards, etc.)
- resolveAsset() handles absolute paths on Windows/Unix via file:// URIs
- RENDERER_FAMILY_MAP synced with actual Remotion compositions
Critical Bug Fixes:
- Windows npx subprocess: run_command() resolves .cmd wrappers via shutil.which()
- Silent renderer downgrade: Remotion failure now returns explicit error with
options instead of silently falling back to FFmpeg
- .env inline comment parsing strips trailing # comments from API keys
- concat_path UnboundLocalError in video_compose finally block
- audio_mixer and showcase_card capture=True kwarg bug
- Selector estimate_cost() calls fixed (_select_tool -> _select_best_tool)
- asset_manifest schema expanded with provider, license, subtype fields
- screen-demo subtitle_gen moved from required to optional tools
- Duration drift detection in post-render final review (>25% warns)
Add green_screen_processor (auto-detect + rembg fallback) and
green_screen_composite (4 layout presets with alpha compositing) tools
to automate the full keying-to-composite pipeline.
Remotion: add AnimatedBackground with gradient mesh and floating orbs
to Explainer, fix caption burn tool (remove entry point arg, auto-detect
dimensions, extend TalkingHead duration to 300s).
Update scene-director, compose-director, and asset-director skill docs
with green screen workflow steps and component constraints.
Scene-director rewrite: agent now watches footage, understands content, and proposes
creative overlays (charts, stats, key terms, comparisons) before building anything.
Presents enhancement plan to user for approval before proceeding.
Compose-director fixes from Round 1 verification:
- eye_enhance: now explicitly required, not silently skippable
- Caption positioning: explicit MarginV=160 for 9:16, never center
- Final encode: mandatory with target file size table
- ASR corrections: new Step 2b to scan transcript and build corrections dict
- Overlay compositing: new Step 3b for burning approved graphics onto footage
Asset-director rewrite: generates Remotion overlay assets (callouts, stat cards,
charts, comparisons) from scene plan. Includes overlay type → Remotion cut mapping
table and dark theme requirements.
Bug fixes found during subagent verification:
- remotion_caption_burn.py: fix run_command API, add npx.cmd for Windows
- visual_qa.py: fix run_command API (3 places), Windows /dev/null → NUL
New tools: face_tracker, visual_qa, eye_enhance, auto_reframe, remotion_caption_burn, showcase_card, silence_cutter. Updated audio_mixer with segmented_music operation and subtitle_gen with ASR corrections. Registered TalkingHead composition in Root.tsx. Rewrote compose/edit/scene director skills for full enhancement chain, Remotion captions, multi-clip assembly, and visual QA. Gitignore cleanup: exclude test demo-props, downloaded music, and generated images.
Add anime_scene rendering engine (AnimeScene + ParticleOverlay components) with multi-image
crossfade, 9 camera motion types, 5 particle systems, and cinematic lighting overlays.
Fix critical Remotion durationInFrames footgun by passing sceneDurationSeconds from parent.
Add audio offset/loop support in Explainer for skipping quiet music intros.
New tools: audio_energy.py analyzes per-second loudness via ebur128 to find optimal music
offset and detect when looping is needed.
Update all 6 animation pipeline skills (proposal, scene, asset, compose, executive-producer,
remotion.md) with battle-tested image_animation workflow including tool availability scan,
FLUX multi-image generation, composition JSON format, pre-render validation, and post-render
self-review.
Add 3 demo compositions (Candyland, Mori no Seishin, Deep Ocean) and anime-ghibli style
playbook. Update README with 3 anime video showcases and animation prompts. Add Animation
Pipeline section to PROMPT_GALLERY.md.
Replace bash-only commands in Makefile with Python equivalents so
setup, test, lint, and clean work on Windows. Add LUMINA video URL
to README embed and note upcoming Ollama/LM Studio local LLM support.
tools/base_tool.py now auto-loads .env at import time so API keys are
available even when tools are imported directly (fixes silent zero-key
fallback in subagents). Added LUMINA bioluminescent terrarium product
ad embed to README. Two new demo compositions: 37 Trillion (one-key
narrated explainer) and LUMINA (music-driven product ad, no narration).
Prove that adding one API key (OPENAI_API_KEY) to the zero-key foundation
produces dramatically better videos for ~$0.69 each. Two compositions built:
The Abyss (deep ocean visual essay) and VOID (neural interface product ad).
New tools:
- audio_probe: ffprobe wrapper with probe_duration() helper
- composition_validator: pre-render checks (asset existence, audio-video sync)
- pixabay_music: royalty-free music scraper (no API key needed)
- freesound_music: Freesound API search + download
Remotion upgrades:
- BackgroundImageLayer: AI images behind data scenes with ken-burns + dark overlay
- Gradient support: all 9 components changed from backgroundColor to background CSS
- CaptionOverlay: word spacing fix, WhisperX word-level subtitles
- HeroTitle: reduced overlay opacity so background images show through
Process codified in agent skills:
- compose-director: audio acquisition flow (present user with voice/music/subtitle
options), mandatory pre-render validation, post-render self-review (extract
frames + transcribe + inspect + present findings to user)
- scene-director: narration duration budgeting (word budget from video duration)
- remotion skill: pre-render validation section
- TTS tool: now returns audio_duration_seconds in result
README updated with VOID product ad video embed.
Fix ComparisonCard and CalloutBox dark-theme passthrough in SceneRenderer.
Add 8 demo-props JSON files (3 fixed originals + 5 new compositions).
Update scene-director and compose-director skills with all 10 Remotion
scene types and zero-key rendering guidance. Document the proven
all-dark-background formula in remotion.md. Add cinematic renderer,
onboarding skill, prompt gallery, and demo render script.
