The selector path previously hid the custom-workflow feature: video_selector
filtered tools on per-operation readiness (bundled WAN models) and both
selectors only chose ToolStatus.AVAILABLE providers, so comfyui_image/
comfyui_video — DEGRADED when bundled model metadata is missing — were
dropped even when the ComfyUI server was up and the caller supplied a full
workflow_json/workflow_path plus output_node.
- Add a custom-workflow readiness path to both selectors: when a custom
workflow is supplied, eligibility is based on server availability (status
!= UNAVAILABLE) for any provider advertising supports.custom_workflow,
not on bundled-model readiness. A custom workflow also restricts routing
to custom-workflow-capable providers, since the graph JSON is ComfyUI
specific.
- Expose workflow_json, workflow_path, output_node, workflow_name,
workflow_model, and workflow_model_stack in both selector schemas so
agents can discover the feature without bypassing the selectors.
- image_selector only forwards the workflow inputs to providers that
declare them.
- Add contract tests for the new eligibility path and schema exposure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removed comfyui_music and its workflow. The ACE-Step model runs in
ComfyUI but the node class names differ across custom node packs
(AceStepModelLoader vs native TextEncodeAceStepAudio, etc.), so a
bundled workflow would break for most users.
Documented the reasoning in the plan doc and listed it as an open
question for future work. Users with ACE-Step working can still use
the workflow_json override on any tool.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Client queries ComfyUI /object_info to discover installed models
(checkpoints, diffusion models, VAE, CLIP, LoRAs)
- Each tool declares its required models and checks them on execute()
- get_status() returns DEGRADED when server is up but models are missing
- Clear error messages tell the user exactly which models to download
- When COMFYUI_SERVER_URL is not set, error message tells the user to
configure it in .env instead of silently failing on localhost:8188
- 8 new tests covering URL config, error messages, and model requirements
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds three new BaseTool providers that delegate GPU work to a running
ComfyUI server via its REST API. This avoids the need to install
PyTorch/diffusers directly, which is critical on hardware where the
ecosystem hasn't caught up (e.g. NVIDIA Blackwell / DGX Spark, aarch64
+ CUDA 13.0).
New files:
- tools/_comfyui/client.py — shared REST client (submit/poll/download)
- tools/_comfyui/workflows/ — 4 bundled workflow templates
- tools/graphics/comfyui_image.py — FLUX 2 Dev NVFP4 text-to-image
- tools/video/comfyui_video.py — WAN 2.2 14B t2v + i2v (4-step LightX2V)
- tools/audio/comfyui_music.py — ACE-Step 3.5B music generation
- tests/contracts/test_comfyui_tools.py — 41 contract tests
- docs/comfyui-adapter-plan.md — design document
Zero changes to existing tools, selectors, registry, or pipelines.
Tools are auto-discovered and selectors pick them up via capability match.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AGENT_GUIDE.md requires the music decision to be made at the proposal stage,
but the only check for the user's music_library/ folder lived in the
asset-director skills, which run later. A user could approve a creative
direction without ever being told a free, intentional music option was sitting
on disk (issue #168).
music_library/ was already referenced as a source_tool in asset artifacts but
had no backing tool. Add a small read-only tool that scans the library folder
(default <project root>/music_library, override via MUSIC_LIBRARY_DIR or a
library_dir input) and lists the audio tracks it finds, with best-effort
durations via ffprobe when present.
Because it inherits BaseTool, the registry auto-discovers it and it appears in
the preflight provider menu alongside music_gen and the stock music sources:
- AVAILABLE when the folder holds at least one audio track
- UNAVAILABLE otherwise, with install_instructions telling the user how to add
tracks
So the user sees their music options before approving creative direction, with
no orchestration code changes. Read-only: no side effects, no cost.
Closes#168
Closes the remaining three atelier audit gaps.
F1 — atelier projects live under repo-root projects/<slug>/ (the
established OpenMontage convention), not under remotion-composer/.
_render_via_atelier now auto-stages source files into
remotion-composer/projects/<slug>/ via mtime-skip copy when the entry
isn't already under the composer tree. Originally tried a directory
junction, but webpack walks up from the entry's REAL location to
resolve node_modules — junctions get dereferenced and module resolution
fails. Copy keeps source-of-truth under projects/, is invisible to the
user's project view, and is cheap (typical project is a few small .tsx
files). Output stays wherever output_path says. effective_entry is
returned in tool data for transparency.
F3 — scripts/scaffold_atelier_project.py generates the engine
boilerplate (index.tsx / Root.tsx / Composition.tsx / art-direction.md /
props template / README) ONLY. No creative content emitted — the
Composition.tsx is a deliberately blank black placeholder with prominent
TODOs. This removes the friction that nudged agents back to templated
mode while making it impossible to ship the scaffold unauthored (review
correctly flags black frames / silent audio as 'revise'). Verified
end-to-end: scaffold → render → review fires the right warnings.
F5 — schemas extended to know about atelier:
- decision_log: 'composition_mode' added to category enum.
- proposal_packet.production_plan: composition_mode + art_direction.
- edit_decisions: composition_mode + bespoke{entry, composition_id,
art_direction, props_path, public_dir, scale, crf, concurrency}.
- final_review.checks: 'atelier' block (stock_reuse_detected,
offending_imports[], art_direction_declared, art_direction).
- final_review.recommended_action: 're_author' added.
- final_review.transcript_comparison: allow null word_accuracy /
transcript_matches_script (pre-existing latent bug surfaced here).
Both pass and fail shapes of the atelier render's final_review now
validate against the updated schema.
Closes the two biggest gaps from the atelier audit: hero bespoke renders
were skipping every automated quality gate, and the templated→atelier
inversion was advisory rather than enforced.
F6 — _render_via_atelier now runs the full _run_final_review (technical
probe + visual spotcheck + audio spotcheck + duration drift + transcript
comparison) plus two atelier-specific doctrine checks:
- stock_reuse_check: rglobs the bespoke project tree for imports from
the stock creative registry (src/components, src/Explainer, etc.).
Any hit fails the render with status="fail" and names the offending
file — the doctrine bypass is now mechanically caught.
- art_direction_check: requires edit_decisions.bespoke.art_direction.
Missing → warning surfaced in final_review.issues_found; forcing
function for step 1 of skills/meta/bespoke-composition.md.
Verified end-to-end on the compound-snowball proof render: positive case
status=pass, atelier checks clean; negative case (deliberate stock import
+ missing art_direction) returns status=fail with both findings named.
F7 — skills/meta/reviewer.md gets a "Composition Authoring Mode Review"
section that enforces the inversion at every stage:
- proposal: composition_mode must be a presented decision with both
options; hero briefs default to atelier; templated for hero requires
an explicit sanctioned reason.
- scene_plan/edit: any stock cut.type in an atelier piece is CRITICAL;
bespoke.{entry, composition_id, art_direction} required.
- compose: consumes final_review.checks.atelier from F6; mandates a
recorded distinctness review (the human taste call the tool can't
automate — its absence is itself CRITICAL).
- publish: blocks ship on any unresolved atelier finding.
Engine knowledge stays reusable; creative components don't.
Introduce a hand-authored, project-local Remotion render path that bypasses
the cut-schema and the stock scene-type registry, for hero/bespoke videos
that must look distinct from one another.
- video_compose: composition_mode="atelier" (or renderer_family="bespoke")
routes to _render_via_atelier, which renders a project-local entry under
remotion-composer/projects/<slug>/ with an optional per-project public_dir
(skips copying the bloated shared public/). No cut-schema, no stock registry.
- skills/meta/bespoke-composition.md: routing skill — art direction
(visual-style) -> motion principles (Disney 12) -> engine mechanics
(remotion-best-practices + stock components read only as a mechanics codex)
-> atelier render. Doctrine: reuse engine knowledge, never creative components.
- AGENT_GUIDE: "Composition Authoring Mode" (templated vs atelier); default
atelier for hero work; scene-type catalog reframed as a mechanics codex.
- animation-runtime-selector + INDEX: authoring-mode-first pointers.
- base_tool.run_command: decode subprocess output as UTF-8/replace (Windows
cp1252 crashed the reader thread on Remotion's Unicode progress output).
- .gitignore: remotion-composer/projects/ (throwaway bespoke compositions).
The compose target resolution was resolved from `profile` (and documented as
overridable via edit_decisions.metadata.compose_target) but the per-segment
scale/pad filter hardcoded 1920x1080. As a result, vertical profiles such as
`tiktok` / `youtube_shorts` / `instagram_reels` silently produced landscape
1920x1080 output instead of 1080x1920 — a silent dimension bug, no error raised.
- Use the resolved target width/height in the per-segment scale/pad filter.
- Implement the previously-stubbed `metadata.compose_target` extension point:
{"width", "height", "fit"} where fit="pad" (letterbox, default, unchanged
behavior) or fit="cover" (scale-to-fill + centre-crop, ideal for vertical).
- Default with no profile/target stays 1920x1080 (backward compatible).
Adds tests/tools/test_video_compose_vertical.py covering default landscape,
profile=tiktok vertical, and compose_target cover override.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
runway (estimate_cost/estimate_runtime/execute) defaulted to gen4_turbo and
higgsfield (execute) defaulted to kling_3.0, while both schemas advertise
seedance_2.0 as model.default. Omitting model under-quoted cost (runway 6x:
$0.25 vs $1.50) and silently generated a different model than advertised,
violating the Decision-Communication / cost-accuracy contract.
Root cause was a default duplicated across schema + 3 methods that drifted.
Collapse it to a single _DEFAULT_MODEL constant referenced everywhere.
Add tests/tools/test_provider_model_defaults.py to lock each tool's
estimate default to its schema default and guard the execute path.
On Windows, passing --props and the JSON path as two separate CLI
arguments causes Remotion to mis-parse the value due to platform quote
escaping, failing with "neither valid JSON nor a file path to a valid
JSON file". Switch to the --props=<path> equals form, which Remotion
recommends for file paths and which works consistently across
platforms.
Fixes#172
- Add get_torch_device() helper in _shared.py: cuda > mps > cpu
- Guard MPS detection for torch builds lacking torch.backends.mps
- Check both is_built() and is_available() for MPS
- Route load_diffusers_pipeline() to resolved device instead of hardcoded cuda
- Use float32 on CPU (float16 is emulated/unreliable), float16 on MPS, bfloat16 on CUDA
- Guard enable_model_cpu_offload() to CUDA-only; fall back to .to(device) on MPS
- Enable attention slicing for MPS memory safety
- Add inspect-based signature guard for device= arg on RealESRGANer/GFPGANer
- Update install_instructions on all LOCAL_GPU tools to mention MPS/Apple Silicon
Google's TTS and Imagen tools advertised service-account auth
(GOOGLE_APPLICATION_CREDENTIALS) but only ever authenticated with an API
key string, so users with a service-account JSON could not use either tool.
google_tts.get_status() also over-reported availability when the JSON was
set, then failed at execute() — a silent-availability bug.
Separately, both hand-rolled _load_dotenv parsers kept inline comments as
values, so after `cp .env.example .env` every keyed tool falsely reported
"available" with no real credentials.
Changes:
- Add tools/google_credentials.py: lazy google-auth Bearer-token helper.
- google_tts: authenticate via Cloud TTS Bearer token when only a service
account is configured; make get_status() honest.
- google_imagen: route service-account auth to Vertex AI
({location}-aiplatform.googleapis.com) with project/location resolution,
alongside the existing AI Studio API-key path.
- Fix both _load_dotenv parsers to strip inline comments (quote-aware).
- Add google-auth to requirements; document the new env vars in .env.example.
- .gitignore: never commit GCP service-account key files.
Verified locally with a real service account: TTS produced a valid MP3 and
Imagen produced a valid 1408x768 PNG via Vertex AI. Existing test suite
passes (2 unrelated pre-existing failures only).
Closes#131
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lines 219 and 227 imported upload_image_fal from _shared but called
the non-existent upload_image_to_fal, raising a NameError whenever
image_to_video or reference_to_video used local image paths.
Adds a transcript_comparison check to VideoCompose._run_final_review
that word-diffs the whisper/whisperx transcript against script.txt and
fails loudly when the TTS engine literally voiced punctuation tokens
(dot, dots, ellipsis, comma, dash, hyphen, period). Chirp3-HD did this
to ellipses in a production run and it slipped past review; now it
cannot.
Also corrects the "tiny background video" gotcha in
skills/core/hyperframes.md. After six renders of blaming HyperFrames
CSS, the real root cause was 640x360 Pexels sources combined with a
fit-and-pad pre-transform — HyperFrames was rendering the letterboxed
input faithfully. Gotcha now walks through the ffprobe diagnostic and
the scale-to-cover fix, and keeps the wrapper-div pattern for the
right reasons (aspect mismatch handling, not framework bug workaround).
Four regression tests cover the new check: punctuation-leak detection,
clean-audio false-positive guard, graceful skip when inputs missing,
and always-present transcript_comparison section.
Separates creative grammar (renderer_family) from technical engine
(render_runtime) so HyperFrames can stand alongside Remotion as a
first-class runtime instead of masquerading as a Remotion sub-case.
Locks runtime choice at proposal stage and enforces it end-to-end: the
schemas require it, video_compose routes by it, the reviewer fails
closed on silent swaps, and a parametrized contract test walks every
pipeline manifest to ensure each planning-stage skill explains the
conversation to the user. Adds hyperframes_compose (scaffold/lint/
validate/render/doctor/add_block), a playbook -> CSS style bridge, and
vendored HyperFrames Layer 3 skills from commit d291358, pinned via
PROVENANCE.md for future re-sync. Final_review now records
render_runtime_used and runtime_swap_detected so compose lies are
catchable after the fact.
Expands the Seedance 2.0 surface from fal.ai-only to every gateway the
OpenMontage toolset can reach, so users on Replicate / Runway /
Higgsfield automatically route cinematic briefs through Seedance 2.0
instead of the gateway's prior default.
- tools/video/seedance_replicate (new): REPLICATE_API_TOKEN adapter for
bytedance/seedance-2.0 and bytedance/seedance-2.0-fast. Same provider
family ("seedance"), quality_score=0.95, premium capability flags.
- tools/video/runway_video: add seedance_2.0 / seedance_2.0_fast to the
model enum as the new default, flag native_audio + multi_shot +
camera_direction + lip_sync in supports, point agent_skills at
seedance-2-0 Layer 3 skill. Note Runway Unlimited/Enterprise + non-US
gating in the model description.
- tools/video/higgsfield_video: same pattern. seedance_2.0 is now the
default model. Cost estimates calibrated to Higgsfield credit pricing
(~$0.50-$1.20 per 5s).
- tools/video/_shared: annotate HEYGEN_PROVIDERS seedance_pro/lite as
Seedance 1.x (2.0 on HeyGen flows through Video Agent / Avatar Shots
which need a separate wrapper — tracked as follow-up).
- .agents/skills/seedance-2-0: update provider matrix to show which
gateways are wrapped (fal.ai, Replicate, Runway, Higgsfield) vs. which
are still roadmap (HeyGen Video Agent, BytePlus, Freepik, aggregators).
The scoring engine already deduplicates by provider="seedance" so
whichever gateway the user has configured wins automatically. Agents
passing preferred_provider="seedance" to video_selector route to the
best available Seedance 2.0 path without naming a specific gateway.
Seedance 2.0 is now routed as the top-ranked video generator whenever a
premium gateway is available. Touches the tool layer, scoring engine,
cinematic pipeline, and both skill layers so discovery works from every
entry point.
- tools/video/seedance_video: BETA stability, quality_score=0.95, add
reference_to_video operation plus 9 img + 3 vid + 3 audio ceilings,
fix pre-existing upload_image_fal import
- tools/base_tool: surface optional quality_score / success_rate /
latency fields in get_info so the scorer can read them
- lib/scoring: fix reliability enum-vs-string bug that was pinning every
available tool to 0.0, switch to overlap coefficient so rich best_for
descriptions aren't penalized, add premium-cinematic feature bonus
- pipeline_defs/cinematic + cinematic asset-director: add pixabay_music
and freesound_music, restore pixabay-first music default
- cinematic compose-director: mandatory Remotion preflight at stage entry
- New Layer 3 .agents/skills/seedance-2-0/SKILL.md (8-part prompt
structure, multi-shot, lip-sync, reference-to-video, provider landscape)
- New Layer 2 skills/creative/prompting/seedance-prompting.md
- Update ai-video-gen, video-gen-prompting, AGENT_GUIDE, INDEX to flag
Seedance 2.0 as the preferred premium default and make the skill
discoverable from every routing path
6 API-based adapters: Pixabay Video, Coverr, NARA (U.S. National Archives),
Library of Congress, Pond5 Public Domain, Videvo. 5 scraper-based adapters
for sites without APIs: ESA, NOAA, Mixkit, Dareful, JAXA. All follow the
StockSource protocol and are auto-discovered — no tool code changes needed.
Updated scene-director, asset-director, and idea-director skills with
source routing guidance so agents know which provider to use for which
content type (e.g. nara for historical, noaa for ocean, esa for space).
Also includes: grok_video capability updates (native audio, 1-15s duration,
new aspect ratios), seedance_video tool (Seedance 2.0 via fal.ai),
ProductReveal Remotion composition registration.
Adds a lightweight alternative to the corpus_builder + clip_search pipeline
that skips CLIP embeddings, motion scores, and index files. Uses the same
StockSource adapter protocol so it works with all providers (Pexels,
Archive.org, NASA, Wikimedia, Unsplash). Asset-director skill now documents
both fast path (direct search) and standard path (corpus + CLIP retrieval).
Pipeline manifest updated to make corpus tools optional.
End-tag now composited over final body footage (ProRes 4444 with alpha)
instead of concatenated as a black card. Updated idea-director,
edit-director, compose-director, and pipeline manifest with overlay-first
instructions. Default Google TTS voice changed from Neural2-D to
Chirp3-HD-Orus across tool code and docs.
All of these changes came out of running the P1 and P2 documentary-montage
audits end-to-end and watching specific things break. Grouping them into
one commit because they share a theme: making the corpus builder and its
stock source adapters robust enough that a real brief can produce a
real corpus without special-casing.
corpus_builder.py + new clip_cache.py + test_clip_cache.py
Phase 1 of the shared-corpus architecture. Adds
~/.openmontage/clips_cache/ — a process-safe, LRU-evicted cache of
downloaded clip files keyed by clip_id. Before each candidate download,
corpus_builder asks the cache whether the bytes already exist on disk
from a previous run; on a hit it hard-links (or copies on cross-drive)
the blob into the caller's corpus dir and skips the network fetch. On
a miss it downloads as usual and ingests the fresh file. Motivation:
re-running the P1 audit after every tool fix was re-downloading gigs
of archive.org footage that had already been fetched in the prior run.
Cache faults never block the pipeline — they degrade gracefully to
normal downloads. The cache bubbles counters into the corpus_builder
return payload so the production report can show hit/miss/bytes-saved.
Default 20 GB cap, overridable via OPENMONTAGE_CACHE_MAX_GB.
Full test coverage: try_link, ingest, stats, LRU eviction, manifest
persistence, lock behavior. 23 tests, tmp_path-scoped.
stock_sources/archive_org.py
1. Three-strategy query cascade (phrase_prox_10 → distinctive_and →
distinctive_or). Motivation: natural-language documentary queries
against archive.org Solr were zeroing out — "1950s family watching
television" returned 0 hits because Solr's default multi-term AND
over-intersects. Walks strict to loose and returns the first
non-empty strategy. Stop words, source hints ("prelinger",
"archive", "footage"), and year tokens ("1950s") are excluded from
the distinctive-token picks since they don't correlate with
Prelinger title tokens.
2. 150 MB per-rendition size cap. archive.org routinely hosts
multi-hundred-megabyte h.264 masters and one 2 GB Prelinger item
poisons corpus build wall-time and disk. Within a format bucket
we now pick the largest rendition under the cap; if nothing fits
we fall through to the next format rather than dropping the item.
3. 180 s default max-duration ceiling when the caller hasn't set one
— archive.org is the only source that routinely hosts feature-
length material and a naive fan-out pulls them into corpora that
only ever want a few seconds per clip.
stock_sources/wikimedia.py
Parallel 3-strategy cascade (full → top2_or → single_best). Motivation:
Commons CirrusSearch also defaults to AND across multi-word queries;
our first P2 diagnostic pass returned 0 video results for 10/10
queries. Same stop-word / source-hint / year-token stripping as
archive_org so the two adapters stay symmetric.
test_stock_source_adapters.py
Rewrote the wikimedia query-builder tests against the new cascade
API. Added coverage for multi-word fallback + source-hint/year
stripping.
video_compose.py
Two small fixes for the Remotion renderer on Windows:
1. Resolve output_path to absolute before invoking the CLI so the
binary can write wherever the caller asked regardless of cwd.
2. Pass cwd=composer_dir to run_command so npx can find the local
Remotion binary under node_modules/.bin. Without this, Windows
npx returns "could not determine executable to run" because it
resolves .bin relative to the process cwd rather than the
script's parent.
New end-to-end pipeline for building thematic documentary montages from
a locally-indexed corpus of free stock footage (Pexels, Archive.org,
NASA). The agent builds a project-local corpus, CLIP-ranks candidates
per scene slot, edits with motion-aware arc logic, and composes via
ffmpeg. No paid APIs required for the full path.
Pipeline definition and director skills:
- pipeline_defs/documentary-montage.yaml: 5-stage manifest
(idea -> scene_plan -> assets -> edit -> compose)
- skills/pipelines/documentary-montage/: 6 director skills
(executive-producer + idea/scene/asset/edit/compose directors)
Corpus and retrieval infrastructure:
- tools/video/corpus_builder.py: multi-source stock fan-out with
resumable append-only corpus index
- tools/video/clip_search.py: CLIP ViT-B/32 retrieval —
rank_for_slot, find_similar_set, diversify, stats
- tools/video/stock_sources/: base + pexels + archive_org + nasa
adapters with a pluggable BaseStockSource contract
- lib/clip_embedder.py: CLIP wrapper
- lib/corpus.py: corpus schema, jsonl append/read, motion-score
caching
video_compose fix rolled in because any concat-based pipeline depends
on it:
- Replace ambiguous -to with -t duration (was double-trimming cuts)
- Force re-encode + normalize to 1920x1080 @ 30fps (was keyframe-
snapping with -c copy and breaking concat on mixed-source corpora)
- Add silent-audio anullsrc fallback for clips without an audio
stream
README: add Documentary Montage row to the pipeline table and bump
the pipeline count from 11 to 12.
- New: Higgsfield video provider with multi-model routing (Kling 3.0, Veo 3.1, Sora 2, WAN 2.5, Soul Cinema) and Soul ID character consistency
- Updated: Runway provider with gen4_aleph and gen3a_turbo models, proper pixel-ratio mapping, probe_output, watermark param, RUNWAYML_API_SECRET env var support
- Docs: Updated provider counts (12→13), tool counts (51→52), added Higgsfield setup/pricing sections across README, AGENT_GUIDE, ARCHITECTURE, and PROVIDERS
New capture layer for the screen-demo pipeline: FFmpeg for quick CLI-driven
recording, Cap integration for polished recordings with webcam overlay and
cursor effects. Selector presents both options and routes based on availability.
Route Chirp and Journey voice families to the v1beta1 API endpoint
automatically. Adds beta voice detection, dynamic API version
selection, and Chirp3-HD cost estimation ($30/1M chars).
Compose-director had contradictory instructions: Step 2 described Remotion
captions/audio, but Steps 5/5b gave detailed FFmpeg code that agents followed
instead. This caused three failures in production: FFmpeg subtitles instead of
Remotion CaptionOverlay, missing audio (mixed externally but never embedded in
Remotion props), and skipped audio verification in post-render review.
Changes:
- compose-director: Remotion is now DEFAULT for audio, captions, text overlays;
FFmpeg is labeled FALLBACK only. Post-render review has mandatory ffprobe gate
and audio transcription with explicit stop conditions.
- remotion.md: routing table updated (captions/audio → Remotion), added universal
Post-Render Verification Protocol for all pipelines (only 2/10 had one).
- scene-director, asset-director: added pitfall for AI-generated text in CTA
screens — must use Remotion text_card for verbatim text.
- image-provider-usage: added Recraft V4 caveat (style param causes 422 on fal.ai).
- recraft_image.py: documented the style parameter 422 issue inline.
The instructions parameter was passed as None when not provided,
causing a 400 error. Now only included when explicitly set. Also
adds speed passthrough support.
Remotion is now the default composition engine for ALL final renders
when available — video clips, images, mixed content. FFmpeg is only
used as fallback when Remotion is not installed or for standalone
operations (trim, transcode). Also fixes three FFmpeg fallback bugs:
profile + copy codec conflict, stream order mapping, and segment
seeking for audio-first containers.
Four gaps found during user acceptance testing of the reference-video
production workflow:
- video_selector: expose aspect_ratio, duration, reference_image_path,
reference_image_url, and image_url in schema so agents can discover
these critical params. Auto-upload local images when the selected
provider requires a URL.
- _shared.py: add upload_image_fal() for local→URL image bridging via
fal.ai storage. Fix upload_image_heygen() to try v2 presigned upload
before falling back to fal.ai (old /v1/asset endpoint returns 404).
- kling_video: call probe_output() so response includes output_path,
duration_seconds, file_size_mb, video dimensions, and codec info.
Video analyzer now uses Farneback dense optical flow to classify each
scene as motion_clip, animated_still, or static_image. This lets the
agent correctly identify whether a reference video uses AI-generated
video clips vs still images with pan/zoom — and plan the right pipeline.
Changes:
- video_analyzer.py: new Step 3b with _classify_scene_motion() and
_read_frame_at() helpers; updated _needs_motion() to use per-scene
motion data instead of pacing heuristic alone
- video-reference-analyst.md: added Motion line to summary template
and instructions to read motion_type field before proposing tools
Implements the 2026-04-02 transformation spec (Phases 1-8) and fixes all
critical bugs found during 5-pipeline E2E testing.
Governance & Decision Intelligence:
- Pipeline-specific stage order in checkpoint (replaces global STAGES list)
- Provider scoring engine (lib/scoring.py) with 7-dimension weighted ranking
- Decision log artifact enforced at proposal/idea stage across all 10 pipelines
- Delivery promise classifier prevents silent motion-to-still downgrades
- Structured shot language in scene_plan schema (camera, lens, lighting, DOF)
- Variation checker and slideshow risk scorer block samey output before render
- Creative intake, capability extension, and creative-intake meta skills
- Final self-review artifact with 5 mandatory checks before presenting output
- Source media review contract for user-supplied footage
Render & Theme System:
- Remotion AnimatedBackground now derives colors from playbook (no more hardcoded
dark blue fintech gradient on every video)
- video_compose builds custom ThemeConfig from playbook YAML colors/fonts —
custom playbooks flow through to Remotion automatically
- Explainer component wires theme to all child components (charts, cards, etc.)
- resolveAsset() handles absolute paths on Windows/Unix via file:// URIs
- RENDERER_FAMILY_MAP synced with actual Remotion compositions
Critical Bug Fixes:
- Windows npx subprocess: run_command() resolves .cmd wrappers via shutil.which()
- Silent renderer downgrade: Remotion failure now returns explicit error with
options instead of silently falling back to FFmpeg
- .env inline comment parsing strips trailing # comments from API keys
- concat_path UnboundLocalError in video_compose finally block
- audio_mixer and showcase_card capture=True kwarg bug
- Selector estimate_cost() calls fixed (_select_tool -> _select_best_tool)
- asset_manifest schema expanded with provider, license, subtype fields
- screen-demo subtitle_gen moved from required to optional tools
- Duration drift detection in post-render final review (>25% warns)
Add green_screen_processor (auto-detect + rembg fallback) and
green_screen_composite (4 layout presets with alpha compositing) tools
to automate the full keying-to-composite pipeline.
Remotion: add AnimatedBackground with gradient mesh and floating orbs
to Explainer, fix caption burn tool (remove entry point arg, auto-detect
dimensions, extend TalkingHead duration to 300s).
Update scene-director, compose-director, and asset-director skill docs
with green screen workflow steps and component constraints.
