video_compose.get_info() reported render_engines.ffmpeg as always
available, unlike the real availability checks used for remotion and
hyperframes. On a machine without ffmpeg on PATH, preflight would
falsely report ffmpeg as usable, letting render_runtime="ffmpeg" get
locked at proposal time only to fail at compose.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
From an independent review of the branch:
- Regression: existing non-raster-but-showable visuals (.svg diagrams) were
dropped by the renderable filter — they were served fine before via <img>
(/thumb passes SVG through). Add .svg to MEDIA_IMAGE_EXT.
- Doc accuracy: the board dedupes decisions by (category, subject), not category
alone (category-only would wrongly merge distinct decisions that share a
category, e.g. TTS vs image provider_selection). Correct AGENT_GUIDE to say
the pair is the key and to reuse the same subject when re-logging.
- Coverage: the new visual-selection logic was untested (which let the earlier
missing-file regression through). Add TestStoryboardVisualSelection covering
the .tsx-animation exclusion, snapshot fallback (exact + <id>_* match), SVG
renderability, the preserved missing-file indicator, and takes = renderable.
Not changed (reviewed, deliberate): .narr clamp at --fs-scale 1.16 degrades
gracefully via the fade + click-to-expand modal; decision dedupe stays keyed on
(category, subject) as the more-correct behavior.
- F-01: cost bar crit (red) state past 90% of budget
- F-02: normalize() hardens fetched board state against sparse payloads
- F-03: /thumb 404s for videos with no extractable poster frame instead
of serving raw video bytes
- F-04: checkpoint artifact path refs only resolve inside the project dir
- F-05 (board half): stall detection — in_progress stage with no disk
activity >10min renders red 'stalled?' + header badge flips to STALLED?;
verified against the real wedged why-cities-glow project
- eval harness from dogfood session committed (visual regression +
interaction smoke, capture watcher, server/gate test suites) +
regression tests for each finding; 46 backlot tests green, visual eval
green (restage-before-capture note logged)
The timeout handling only took effect on a direct _remotion_render() call. The
high-level execute(operation='render') path goes through _render(), which builds
a fresh remotion_inputs dict (edit_decisions, output_path, profile) and dropped
remotion_timeout_ms — so callers of the documented operation='render' path never
got the timeout passed to the Remotion CLI. Forward it there.
Adds a test exercising _render() (not just _remotion_render()) to cover the
high-level forwarding path.
Refs #217
The high-level Remotion render path hid the useful failure reason. run_command
runs with check=True + capture_output, so a non-zero exit raised
CalledProcessError whose str() is only 'returned non-zero exit status 1' — the
actual Remotion diagnostics in stderr were dropped. Catch CalledProcessError
and surface the stderr/stdout tail, and TimeoutExpired with an actionable hint.
Also add a creator-facing remotion_timeout_ms input, passed through as
Remotion's --timeout (headless-browser setup + delayRender). Slow browser
startup on restricted networks previously failed opaquely at the default 30s
with no way to raise it. The subprocess timeout is widened to match so
run_command does not kill Remotion before its own timeout fires.
Closes#217
- backlot/state.py: BoardState from disk — stage rail with gate audit
(gate_skipped detection from history/), scene_plan x script x
asset_manifest storyboard join, takes, generating-state from events,
media discovery incl. atelier root-render heuristic, degradation ladder,
library summaries
- backlot/server.py: FastAPI on 4750 — /api/projects, /api/project/{id}/state,
SSE change feeds (project + library) fed by a watchfiles watcher,
/media with range support and traversal protection, UI mounts
- backlot/__main__.py: 'python -m backlot open [project]' idempotent
launcher (spawns detached server, opens browser); 'serve' foreground
- verified against real projects: 73 listed, full state for
signal-from-tomorrow, 206 range responses, SSE change push on
filesystem write
Address PR #240 review feedback from @calesthio:
1. dashscope_image: save EVERY returned image URL, not just the first.
The tool advertised multiple_outputs and accepted n>1 but only read
content[0], silently dropping paid outputs. Now collects all image
URLs across choices/content and downloads each to a distinct indexed
path (foo.png -> foo_1.png, foo_2.png, ...). images_generated now
reflects the actual count downloaded.
Per Qwen Cloud docs, a multi-output task is SUCCEEDED if at least one
image is generated; choices with finish_reason != "stop" are skipped
to avoid downloading partial/failed results.
2. Complete idempotency_key_fields so different requests no longer
collide and reuse stale artifacts:
- image: + negative_prompt, seed, prompt_extend, watermark
- tts: + instructions
- asr: + enable_words, language_hints
Adds 19 regression tests (114 total, all pass, no API keys needed):
- TestDashscopeImageMultiOutput: URL extraction across choices / within
one choice / failed-choice skipping, path resolution for
single/multi/no-extension, end-to-end multi-image download with a
mocked 3-URL DashScope response verifying all 3 files land on disk,
single-image legacy path behavior
- TestDashscopeIdempotencyKeys: field presence + key-differs-on-value
for every newly added field across all three tools
Per maintainer feedback on PR #227:
- Revert success=not issues back to success=True — tool execution
succeeded even when QA finds issues; verdict lives in status/issues
- Update test to assert the real contract: success=True + status='revise'
+ issues non-empty, matching how compose-director actually gates
- Consistent with visual_qa.py: success=True, verdict in validation_passed
Addresses review feedback on export_bundle:
- If subtitles_path or thumbnail_path is provided but the file is missing, the
tool now fails with an explicit error instead of silently producing a package
without that asset (which could ship an approved deliverable missing part of
its content).
- Default export location now stays inside the project workspace: when the
render lives at projects/<name>/renders/..., the bundle defaults to
projects/<name>/exports/ (alongside artifacts/, assets/, renders/) rather than
a repo-root exports/<name>/. export_dir remains an explicit override.
Tests cover both: missing optional asset errors, and the project-workspace
default path.
The selector path previously hid the custom-workflow feature: video_selector
filtered tools on per-operation readiness (bundled WAN models) and both
selectors only chose ToolStatus.AVAILABLE providers, so comfyui_image/
comfyui_video — DEGRADED when bundled model metadata is missing — were
dropped even when the ComfyUI server was up and the caller supplied a full
workflow_json/workflow_path plus output_node.
- Add a custom-workflow readiness path to both selectors: when a custom
workflow is supplied, eligibility is based on server availability (status
!= UNAVAILABLE) for any provider advertising supports.custom_workflow,
not on bundled-model readiness. A custom workflow also restricts routing
to custom-workflow-capable providers, since the graph JSON is ComfyUI
specific.
- Expose workflow_json, workflow_path, output_node, workflow_name,
workflow_model, and workflow_model_stack in both selector schemas so
agents can discover the feature without bypassing the selectors.
- image_selector only forwards the workflow inputs to providers that
declare them.
- Add contract tests for the new eligibility path and schema exposure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removed comfyui_music and its workflow. The ACE-Step model runs in
ComfyUI but the node class names differ across custom node packs
(AceStepModelLoader vs native TextEncodeAceStepAudio, etc.), so a
bundled workflow would break for most users.
Documented the reasoning in the plan doc and listed it as an open
question for future work. Users with ACE-Step working can still use
the workflow_json override on any tool.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Client queries ComfyUI /object_info to discover installed models
(checkpoints, diffusion models, VAE, CLIP, LoRAs)
- Each tool declares its required models and checks them on execute()
- get_status() returns DEGRADED when server is up but models are missing
- Clear error messages tell the user exactly which models to download
- When COMFYUI_SERVER_URL is not set, error message tells the user to
configure it in .env instead of silently failing on localhost:8188
- 8 new tests covering URL config, error messages, and model requirements
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds three new BaseTool providers that delegate GPU work to a running
ComfyUI server via its REST API. This avoids the need to install
PyTorch/diffusers directly, which is critical on hardware where the
ecosystem hasn't caught up (e.g. NVIDIA Blackwell / DGX Spark, aarch64
+ CUDA 13.0).
New files:
- tools/_comfyui/client.py — shared REST client (submit/poll/download)
- tools/_comfyui/workflows/ — 4 bundled workflow templates
- tools/graphics/comfyui_image.py — FLUX 2 Dev NVFP4 text-to-image
- tools/video/comfyui_video.py — WAN 2.2 14B t2v + i2v (4-step LightX2V)
- tools/audio/comfyui_music.py — ACE-Step 3.5B music generation
- tests/contracts/test_comfyui_tools.py — 41 contract tests
- docs/comfyui-adapter-plan.md — design document
Zero changes to existing tools, selectors, registry, or pipelines.
Tools are auto-discovered and selectors pick them up via capability match.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The contract test in test_character_animation_pipeline.py asserts
result.success after the QA run, implying success should reflect
whether QA passed. But CharacterAnimationReviewer always returned
success=True even when issues were found and status was 'revise'.
This creates a silent failure path: callers that gate on result.success
(the standard ToolResult contract) would treat a broken rig as a clean
pass without ever reading report['status'].
Fix: return success=not issues so result.success is False when QA finds
problems, consistent with the test contract and with how execution errors
are already handled (success=False on exception paths).
Also adds test_character_reviewer_success_false_when_qa_finds_issues to
explicitly assert this contract — a broken rig with missing joints must
produce status='revise', non-empty issues[], and success=False.
Every pipeline ends in a publish stage that produces a publish_log artifact,
but tools/publishers/ shipped empty (only __init__.py) — no tool backed the
PUBLISH tier, so the mechanical packaging (copy the render, write metadata
files, lay out the export directory, emit a schema-valid publish_log) had to be
hand-rolled by the agent every run.
Add a local, offline export bundler:
- capability 'publish', provider 'local', runtime 'local', deterministic, no cost
- takes the final video_path plus the SEO metadata the publish-director prepares
(title, description, tags, hashtags, chapters, optional subtitles/thumbnail)
- writes exports/<project>/{video,metadata,thumbnails}/ matching the
publish-director skill's documented layout
- returns a schema-valid publish_log (status: 'exported') in data, validated
against schemas/artifacts/publish_log before returning so a bad entry fails
here rather than at checkpoint
It does not upload — a networked publisher (e.g. YouTube) can be added later as
a separate provider under the same 'publish' capability.
Tests cover the contract, registry discovery, the export layout, chapter-time
formatting, the schema-valid publish_log, and the missing-video error path.
AGENT_GUIDE.md requires the music decision to be made at the proposal stage,
but the only check for the user's music_library/ folder lived in the
asset-director skills, which run later. A user could approve a creative
direction without ever being told a free, intentional music option was sitting
on disk (issue #168).
music_library/ was already referenced as a source_tool in asset artifacts but
had no backing tool. Add a small read-only tool that scans the library folder
(default <project root>/music_library, override via MUSIC_LIBRARY_DIR or a
library_dir input) and lists the audio tracks it finds, with best-effort
durations via ffprobe when present.
Because it inherits BaseTool, the registry auto-discovers it and it appears in
the preflight provider menu alongside music_gen and the stock music sources:
- AVAILABLE when the folder holds at least one audio track
- UNAVAILABLE otherwise, with install_instructions telling the user how to add
tracks
So the user sees their music options before approving creative direction, with
no orchestration code changes. Read-only: no side effects, no cost.
Closes#168
The compose target resolution was resolved from `profile` (and documented as
overridable via edit_decisions.metadata.compose_target) but the per-segment
scale/pad filter hardcoded 1920x1080. As a result, vertical profiles such as
`tiktok` / `youtube_shorts` / `instagram_reels` silently produced landscape
1920x1080 output instead of 1080x1920 — a silent dimension bug, no error raised.
- Use the resolved target width/height in the per-segment scale/pad filter.
- Implement the previously-stubbed `metadata.compose_target` extension point:
{"width", "height", "fit"} where fit="pad" (letterbox, default, unchanged
behavior) or fit="cover" (scale-to-fill + centre-crop, ideal for vertical).
- Default with no profile/target stays 1920x1080 (backward compatible).
Adds tests/tools/test_video_compose_vertical.py covering default landscape,
profile=tiktok vertical, and compose_target cover override.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
runway (estimate_cost/estimate_runtime/execute) defaulted to gen4_turbo and
higgsfield (execute) defaulted to kling_3.0, while both schemas advertise
seedance_2.0 as model.default. Omitting model under-quoted cost (runway 6x:
$0.25 vs $1.50) and silently generated a different model than advertised,
violating the Decision-Communication / cost-accuracy contract.
Root cause was a default duplicated across schema + 3 methods that drifted.
Collapse it to a single _DEFAULT_MODEL constant referenced everywhere.
Add tests/tools/test_provider_model_defaults.py to lock each tool's
estimate default to its schema default and guard the execute path.
tests/qa/test_08_end_to_end.py runs at module import, so pytest fails
collection with CheckpointValidationError: the Stage 5 edit_decisions
fixture omits render_runtime, which edit_decisions.schema.json lists as
required. Per AGENT_GUIDE the runtime is locked at proposal and carried
through edit unchanged, so the fixture now sources it from the same
proposal_packet["production_plan"]["render_runtime"] ("remotion")
instead of hardcoding an unrelated value, keeping the fixture internally
consistent with the proposal it builds on.
Adds a transcript_comparison check to VideoCompose._run_final_review
that word-diffs the whisper/whisperx transcript against script.txt and
fails loudly when the TTS engine literally voiced punctuation tokens
(dot, dots, ellipsis, comma, dash, hyphen, period). Chirp3-HD did this
to ellipses in a production run and it slipped past review; now it
cannot.
Also corrects the "tiny background video" gotcha in
skills/core/hyperframes.md. After six renders of blaming HyperFrames
CSS, the real root cause was 640x360 Pexels sources combined with a
fit-and-pad pre-transform — HyperFrames was rendering the letterboxed
input faithfully. Gotcha now walks through the ffprobe diagnostic and
the scale-to-cover fix, and keeps the wrapper-div pattern for the
right reasons (aspect mismatch handling, not framework bug workaround).
Four regression tests cover the new check: punctuation-leak detection,
clean-audio false-positive guard, graceful skip when inputs missing,
and always-present transcript_comparison section.
Separates creative grammar (renderer_family) from technical engine
(render_runtime) so HyperFrames can stand alongside Remotion as a
first-class runtime instead of masquerading as a Remotion sub-case.
Locks runtime choice at proposal stage and enforces it end-to-end: the
schemas require it, video_compose routes by it, the reviewer fails
closed on silent swaps, and a parametrized contract test walks every
pipeline manifest to ensure each planning-stage skill explains the
conversation to the user. Adds hyperframes_compose (scaffold/lint/
validate/render/doctor/add_block), a playbook -> CSS style bridge, and
vendored HyperFrames Layer 3 skills from commit d291358, pinned via
PROVENANCE.md for future re-sync. Final_review now records
render_runtime_used and runtime_swap_detected so compose lies are
catchable after the fact.
All of these changes came out of running the P1 and P2 documentary-montage
audits end-to-end and watching specific things break. Grouping them into
one commit because they share a theme: making the corpus builder and its
stock source adapters robust enough that a real brief can produce a
real corpus without special-casing.
corpus_builder.py + new clip_cache.py + test_clip_cache.py
Phase 1 of the shared-corpus architecture. Adds
~/.openmontage/clips_cache/ — a process-safe, LRU-evicted cache of
downloaded clip files keyed by clip_id. Before each candidate download,
corpus_builder asks the cache whether the bytes already exist on disk
from a previous run; on a hit it hard-links (or copies on cross-drive)
the blob into the caller's corpus dir and skips the network fetch. On
a miss it downloads as usual and ingests the fresh file. Motivation:
re-running the P1 audit after every tool fix was re-downloading gigs
of archive.org footage that had already been fetched in the prior run.
Cache faults never block the pipeline — they degrade gracefully to
normal downloads. The cache bubbles counters into the corpus_builder
return payload so the production report can show hit/miss/bytes-saved.
Default 20 GB cap, overridable via OPENMONTAGE_CACHE_MAX_GB.
Full test coverage: try_link, ingest, stats, LRU eviction, manifest
persistence, lock behavior. 23 tests, tmp_path-scoped.
stock_sources/archive_org.py
1. Three-strategy query cascade (phrase_prox_10 → distinctive_and →
distinctive_or). Motivation: natural-language documentary queries
against archive.org Solr were zeroing out — "1950s family watching
television" returned 0 hits because Solr's default multi-term AND
over-intersects. Walks strict to loose and returns the first
non-empty strategy. Stop words, source hints ("prelinger",
"archive", "footage"), and year tokens ("1950s") are excluded from
the distinctive-token picks since they don't correlate with
Prelinger title tokens.
2. 150 MB per-rendition size cap. archive.org routinely hosts
multi-hundred-megabyte h.264 masters and one 2 GB Prelinger item
poisons corpus build wall-time and disk. Within a format bucket
we now pick the largest rendition under the cap; if nothing fits
we fall through to the next format rather than dropping the item.
3. 180 s default max-duration ceiling when the caller hasn't set one
— archive.org is the only source that routinely hosts feature-
length material and a naive fan-out pulls them into corpora that
only ever want a few seconds per clip.
stock_sources/wikimedia.py
Parallel 3-strategy cascade (full → top2_or → single_best). Motivation:
Commons CirrusSearch also defaults to AND across multi-word queries;
our first P2 diagnostic pass returned 0 video results for 10/10
queries. Same stop-word / source-hint / year-token stripping as
archive_org so the two adapters stay symmetric.
test_stock_source_adapters.py
Rewrote the wikimedia query-builder tests against the new cascade
API. Added coverage for multi-word fallback + source-hint/year
stripping.
video_compose.py
Two small fixes for the Remotion renderer on Windows:
1. Resolve output_path to absolute before invoking the CLI so the
binary can write wherever the caller asked regardless of cwd.
2. Pass cwd=composer_dir to run_command so npx can find the local
Remotion binary under node_modules/.bin. Without this, Windows
npx returns "could not determine executable to run" because it
resolves .bin relative to the process cwd rather than the
script's parent.
test_01_tts.py (ElevenLabs), test_02_image_gen.py (DALL-E + FLUX),
and test_03_music.py (ElevenLabs Music) were burning ~$0.44 per run
against live API keys. These were run repeatedly and exhausted the
ElevenLabs starter plan quota (30K characters).
Implements the 2026-04-02 transformation spec (Phases 1-8) and fixes all
critical bugs found during 5-pipeline E2E testing.
Governance & Decision Intelligence:
- Pipeline-specific stage order in checkpoint (replaces global STAGES list)
- Provider scoring engine (lib/scoring.py) with 7-dimension weighted ranking
- Decision log artifact enforced at proposal/idea stage across all 10 pipelines
- Delivery promise classifier prevents silent motion-to-still downgrades
- Structured shot language in scene_plan schema (camera, lens, lighting, DOF)
- Variation checker and slideshow risk scorer block samey output before render
- Creative intake, capability extension, and creative-intake meta skills
- Final self-review artifact with 5 mandatory checks before presenting output
- Source media review contract for user-supplied footage
Render & Theme System:
- Remotion AnimatedBackground now derives colors from playbook (no more hardcoded
dark blue fintech gradient on every video)
- video_compose builds custom ThemeConfig from playbook YAML colors/fonts —
custom playbooks flow through to Remotion automatically
- Explainer component wires theme to all child components (charts, cards, etc.)
- resolveAsset() handles absolute paths on Windows/Unix via file:// URIs
- RENDERER_FAMILY_MAP synced with actual Remotion compositions
Critical Bug Fixes:
- Windows npx subprocess: run_command() resolves .cmd wrappers via shutil.which()
- Silent renderer downgrade: Remotion failure now returns explicit error with
options instead of silently falling back to FFmpeg
- .env inline comment parsing strips trailing # comments from API keys
- concat_path UnboundLocalError in video_compose finally block
- audio_mixer and showcase_card capture=True kwarg bug
- Selector estimate_cost() calls fixed (_select_tool -> _select_best_tool)
- asset_manifest schema expanded with provider, license, subtype fields
- screen-demo subtitle_gen moved from required to optional tools
- Duration drift detection in post-render final review (>25% warns)
Two new provider tools following the BaseTool pattern with auto-discovery:
- google_imagen: Imagen 4 image generation via Generative Language REST API
- google_tts: Google Cloud TTS with 700+ voices across 50+ languages
Both share GOOGLE_API_KEY env var. Selectors auto-discover them — no
selector code changes needed. Docs and contract tests updated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
