`_segmented_music` mixed the video's audio with the shaped music via
`amix=inputs=2`, whose default `normalize=1` scales every input by 1/inputs
(x0.5, -6 dB). Unlike `_mix` and `_full_mix`, this path has no `loudnorm` stage
afterward to re-normalize, so the narration was permanently attenuated across
the entire timeline — including the stretches where the music volume expression
evaluates to 0. A one-second music segment quietly dropped the narration by
~6 dB for the whole video.
Add `normalize=0` to the amix: the music is already scaled to `music_volume`
by the `volume` expression, so speech passes at unity. Verified with ffmpeg —
narration in a no-music region tracks the stereo/aac conversion baseline
instead of sitting 6 dB below it.
The tool advertised `multiple_outputs: True`, accepted `n` (1-4) in its schema,
requested `n` images from the API, and scaled `estimate_cost` by `n` — but the
result handling was hardcoded to `response.data[0]`. Images 1..n-1 were decoded
never, written never, and absent from `artifacts`, so a caller who set `n=4`
paid for four images and received one.
Iterate over `response.data`, writing each image to a distinct path (suffixed
`_1`, `_2`, … when several are requested, mirroring `grok_image` /
`dashscope_image`), and return `outputs` / `images_generated` alongside the
full `artifacts` list. A single image keeps its exact requested path.
The bundled wan22-t2v-4step.json workflow loads the 14B FP8 diffusion
pair (wan2.2_t2v_high/low_noise_14B_fp8_scaled.safetensors), which
produce 16-channel latents, but its VAELoader referenced
wan2.2_vae.safetensors — the WAN 2.2 5B model's VAE, which expects
48-channel latents. Every T2V run therefore failed at VAEDecode with:
Expected tensor to have size 48 at dimension 1, but got size 16
Switch the workflow to wan_2.1_vae.safetensors, matching the 14B
models and the sibling wan22-i2v-4step.json, and update the T2V
required-models list in tools/video/comfyui_video.py to match so
preflight checks for the VAE that is actually used.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
video_compose.get_info() reported render_engines.ffmpeg as always
available, unlike the real availability checks used for remotion and
hyperframes. On a machine without ffmpeg on PATH, preflight would
falsely report ffmpeg as usable, letting render_runtime="ffmpeg" get
locked at proposal time only to fail at compose.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The prior dunder denylist was still bypassable via print.__self__ (the builtins
module) -> .open(...), reachable with no import and no bare open/__builtins__/
getattr name. Enumerating dangerous dunders is whack-a-mole, so block ALL
dunder attribute access generically and allow only the tiny set legitimate
scenes need (super().__init__, occasional Type.__name__). This closes the
print.__self__ / .__class__ / .__globals__ introspection-escape class at once.
Static analysis still has a ceiling — a real subprocess sandbox is the complete
fix — but the default path no longer executes the reported secret-read payloads.
Adds regression tests for print.__self__ and for super().__init__ staying allowed.
Refs #219
The scan only flagged dangerous builtins as direct call targets (ast.Name func)
and dunders as attribute access, so it missed indirection like
`__builtins__['open']('.env').read()` and `getattr(o, '__class__')` — the
default path still executed secret-reading code.
Block dangerous identifiers wherever they appear as a bare name (open, eval,
exec, compile, __import__, __builtins__, getattr/setattr/delattr, globals/
locals/vars) rather than only as a call target, and extend the blocked dunder
set (__class__, __dict__, __getattribute__, __reduce__, ...). This closes the
reported no-import bypass while genuine math scenes still pass.
Still defense-in-depth, not a full sandbox; the allow_unsafe_code opt-out and
explicit code-execution contract remain. A subprocess-level sandbox is the
right follow-up for complete isolation.
Refs #219
math_animate writes caller-supplied Python to scene.py and runs Manim on it —
arbitrary local code execution with no boundary surfaced in the tool contract.
In an agent-driven system the scene_code may be LLM-generated or influenced by
untrusted prompt content, so import-time code or construct() could read
secrets/SSH material, open network connections, or spawn subprocesses.
Add a static AST safety scan that rejects dangerous imports (os, subprocess,
socket, requests, ctypes, ...), dangerous builtins (eval/exec/compile/open/
__import__), and sandbox-escape dunders (__globals__, __subclasses__, ...)
before Manim runs. Genuine math scenes (manim, numpy, math, ...) pass
untouched. This is defense-in-depth, not a sandbox: a determined attacker can
evade a static denylist, so it is paired with an explicit allow_unsafe_code
opt-out and a tool contract (schema + side_effects) that names the boundary.
Closes#219
The prior fix removed the dangling pad but still reused the speech filter
output for two consumers (sidechain key + final mix). FFmpeg auto-splits a
reused *input* label on some builds (macOS) but the Linux ffmpeg on CI rejects
it, so both full_mix ducking tests failed there.
Build a single [speech_all] stream and asplit it into [speech_key] (sidechain
key) and [speech_out] (final mix) so every filter label is produced once and
consumed once. Verified the generated graph for the single- and multi-narration
cases: no label is consumed more than once.
Refs #265
full_mix with ducking enabled (the default) failed for a single narration
track + one music bed — the most common shape — because the ducking branch
appended an acopy[speech_dup] filter whose output pad was never consumed,
leaving the filtergraph with a dangling output that ffmpeg rejects.
For a single speech track speech_out is '[a0]' (starts with '[a'), so the
guarded append fired; the compensating pop() only removes the empty-string
case from the multi-speech branch, so the dead pad survived exactly in the
single-narration case. The speech stream is already re-derived for the final
mix via [speech_out], and ffmpeg auto-splits the reused input label, so the
duplicate is unnecessary. Multi-speech and SFX paths are unaffected.
Adds regression tests for single- and multi-narration full_mix with ducking.
Closes#265
The timeout handling only took effect on a direct _remotion_render() call. The
high-level execute(operation='render') path goes through _render(), which builds
a fresh remotion_inputs dict (edit_decisions, output_path, profile) and dropped
remotion_timeout_ms — so callers of the documented operation='render' path never
got the timeout passed to the Remotion CLI. Forward it there.
Adds a test exercising _render() (not just _remotion_render()) to cover the
high-level forwarding path.
Refs #217
The high-level Remotion render path hid the useful failure reason. run_command
runs with check=True + capture_output, so a non-zero exit raised
CalledProcessError whose str() is only 'returned non-zero exit status 1' — the
actual Remotion diagnostics in stderr were dropped. Catch CalledProcessError
and surface the stderr/stdout tail, and TimeoutExpired with an actionable hint.
Also add a creator-facing remotion_timeout_ms input, passed through as
Remotion's --timeout (headless-browser setup + delayRender). Slow browser
startup on restricted networks previously failed opaquely at the default 30s
with no way to raise it. The subprocess timeout is widened to match so
run_command does not kill Remotion before its own timeout fires.
Closes#217
Address PR #240 review feedback from @calesthio:
1. dashscope_image: save EVERY returned image URL, not just the first.
The tool advertised multiple_outputs and accepted n>1 but only read
content[0], silently dropping paid outputs. Now collects all image
URLs across choices/content and downloads each to a distinct indexed
path (foo.png -> foo_1.png, foo_2.png, ...). images_generated now
reflects the actual count downloaded.
Per Qwen Cloud docs, a multi-output task is SUCCEEDED if at least one
image is generated; choices with finish_reason != "stop" are skipped
to avoid downloading partial/failed results.
2. Complete idempotency_key_fields so different requests no longer
collide and reuse stale artifacts:
- image: + negative_prompt, seed, prompt_extend, watermark
- tts: + instructions
- asr: + enable_words, language_hints
Adds 19 regression tests (114 total, all pass, no API keys needed):
- TestDashscopeImageMultiOutput: URL extraction across choices / within
one choice / failed-choice skipping, path resolution for
single/multi/no-extension, end-to-end multi-image download with a
mocked 3-URL DashScope response verifying all 3 files land on disk,
single-image legacy path behavior
- TestDashscopeIdempotencyKeys: field presence + key-differs-on-value
for every newly added field across all three tools
Addresses review feedback on export_bundle:
- If subtitles_path or thumbnail_path is provided but the file is missing, the
tool now fails with an explicit error instead of silently producing a package
without that asset (which could ship an approved deliverable missing part of
its content).
- Default export location now stays inside the project workspace: when the
render lives at projects/<name>/renders/..., the bundle defaults to
projects/<name>/exports/ (alongside artifacts/, assets/, renders/) rather than
a repo-root exports/<name>/. export_dir remains an explicit override.
Tests cover both: missing optional asset errors, and the project-workspace
default path.
The selector path previously hid the custom-workflow feature: video_selector
filtered tools on per-operation readiness (bundled WAN models) and both
selectors only chose ToolStatus.AVAILABLE providers, so comfyui_image/
comfyui_video — DEGRADED when bundled model metadata is missing — were
dropped even when the ComfyUI server was up and the caller supplied a full
workflow_json/workflow_path plus output_node.
- Add a custom-workflow readiness path to both selectors: when a custom
workflow is supplied, eligibility is based on server availability (status
!= UNAVAILABLE) for any provider advertising supports.custom_workflow,
not on bundled-model readiness. A custom workflow also restricts routing
to custom-workflow-capable providers, since the graph JSON is ComfyUI
specific.
- Expose workflow_json, workflow_path, output_node, workflow_name,
workflow_model, and workflow_model_stack in both selector schemas so
agents can discover the feature without bypassing the selectors.
- image_selector only forwards the workflow inputs to providers that
declare them.
- Add contract tests for the new eligibility path and schema exposure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removed comfyui_music and its workflow. The ACE-Step model runs in
ComfyUI but the node class names differ across custom node packs
(AceStepModelLoader vs native TextEncodeAceStepAudio, etc.), so a
bundled workflow would break for most users.
Documented the reasoning in the plan doc and listed it as an open
question for future work. Users with ACE-Step working can still use
the workflow_json override on any tool.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Client queries ComfyUI /object_info to discover installed models
(checkpoints, diffusion models, VAE, CLIP, LoRAs)
- Each tool declares its required models and checks them on execute()
- get_status() returns DEGRADED when server is up but models are missing
- Clear error messages tell the user exactly which models to download
- When COMFYUI_SERVER_URL is not set, error message tells the user to
configure it in .env instead of silently failing on localhost:8188
- 8 new tests covering URL config, error messages, and model requirements
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds three new BaseTool providers that delegate GPU work to a running
ComfyUI server via its REST API. This avoids the need to install
PyTorch/diffusers directly, which is critical on hardware where the
ecosystem hasn't caught up (e.g. NVIDIA Blackwell / DGX Spark, aarch64
+ CUDA 13.0).
New files:
- tools/_comfyui/client.py — shared REST client (submit/poll/download)
- tools/_comfyui/workflows/ — 4 bundled workflow templates
- tools/graphics/comfyui_image.py — FLUX 2 Dev NVFP4 text-to-image
- tools/video/comfyui_video.py — WAN 2.2 14B t2v + i2v (4-step LightX2V)
- tools/audio/comfyui_music.py — ACE-Step 3.5B music generation
- tests/contracts/test_comfyui_tools.py — 41 contract tests
- docs/comfyui-adapter-plan.md — design document
Zero changes to existing tools, selectors, registry, or pipelines.
Tools are auto-discovered and selectors pick them up via capability match.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Every pipeline ends in a publish stage that produces a publish_log artifact,
but tools/publishers/ shipped empty (only __init__.py) — no tool backed the
PUBLISH tier, so the mechanical packaging (copy the render, write metadata
files, lay out the export directory, emit a schema-valid publish_log) had to be
hand-rolled by the agent every run.
Add a local, offline export bundler:
- capability 'publish', provider 'local', runtime 'local', deterministic, no cost
- takes the final video_path plus the SEO metadata the publish-director prepares
(title, description, tags, hashtags, chapters, optional subtitles/thumbnail)
- writes exports/<project>/{video,metadata,thumbnails}/ matching the
publish-director skill's documented layout
- returns a schema-valid publish_log (status: 'exported') in data, validated
against schemas/artifacts/publish_log before returning so a bad entry fails
here rather than at checkpoint
It does not upload — a networked publisher (e.g. YouTube) can be added later as
a separate provider under the same 'publish' capability.
Tests cover the contract, registry discovery, the export layout, chapter-time
formatting, the schema-valid publish_log, and the missing-video error path.
AGENT_GUIDE.md requires the music decision to be made at the proposal stage,
but the only check for the user's music_library/ folder lived in the
asset-director skills, which run later. A user could approve a creative
direction without ever being told a free, intentional music option was sitting
on disk (issue #168).
music_library/ was already referenced as a source_tool in asset artifacts but
had no backing tool. Add a small read-only tool that scans the library folder
(default <project root>/music_library, override via MUSIC_LIBRARY_DIR or a
library_dir input) and lists the audio tracks it finds, with best-effort
durations via ffprobe when present.
Because it inherits BaseTool, the registry auto-discovers it and it appears in
the preflight provider menu alongside music_gen and the stock music sources:
- AVAILABLE when the folder holds at least one audio track
- UNAVAILABLE otherwise, with install_instructions telling the user how to add
tracks
So the user sees their music options before approving creative direction, with
no orchestration code changes. Read-only: no side effects, no cost.
Closes#168
Closes the remaining three atelier audit gaps.
F1 — atelier projects live under repo-root projects/<slug>/ (the
established OpenMontage convention), not under remotion-composer/.
_render_via_atelier now auto-stages source files into
remotion-composer/projects/<slug>/ via mtime-skip copy when the entry
isn't already under the composer tree. Originally tried a directory
junction, but webpack walks up from the entry's REAL location to
resolve node_modules — junctions get dereferenced and module resolution
fails. Copy keeps source-of-truth under projects/, is invisible to the
user's project view, and is cheap (typical project is a few small .tsx
files). Output stays wherever output_path says. effective_entry is
returned in tool data for transparency.
F3 — scripts/scaffold_atelier_project.py generates the engine
boilerplate (index.tsx / Root.tsx / Composition.tsx / art-direction.md /
props template / README) ONLY. No creative content emitted — the
Composition.tsx is a deliberately blank black placeholder with prominent
TODOs. This removes the friction that nudged agents back to templated
mode while making it impossible to ship the scaffold unauthored (review
correctly flags black frames / silent audio as 'revise'). Verified
end-to-end: scaffold → render → review fires the right warnings.
F5 — schemas extended to know about atelier:
- decision_log: 'composition_mode' added to category enum.
- proposal_packet.production_plan: composition_mode + art_direction.
- edit_decisions: composition_mode + bespoke{entry, composition_id,
art_direction, props_path, public_dir, scale, crf, concurrency}.
- final_review.checks: 'atelier' block (stock_reuse_detected,
offending_imports[], art_direction_declared, art_direction).
- final_review.recommended_action: 're_author' added.
- final_review.transcript_comparison: allow null word_accuracy /
transcript_matches_script (pre-existing latent bug surfaced here).
Both pass and fail shapes of the atelier render's final_review now
validate against the updated schema.
Closes the two biggest gaps from the atelier audit: hero bespoke renders
were skipping every automated quality gate, and the templated→atelier
inversion was advisory rather than enforced.
F6 — _render_via_atelier now runs the full _run_final_review (technical
probe + visual spotcheck + audio spotcheck + duration drift + transcript
comparison) plus two atelier-specific doctrine checks:
- stock_reuse_check: rglobs the bespoke project tree for imports from
the stock creative registry (src/components, src/Explainer, etc.).
Any hit fails the render with status="fail" and names the offending
file — the doctrine bypass is now mechanically caught.
- art_direction_check: requires edit_decisions.bespoke.art_direction.
Missing → warning surfaced in final_review.issues_found; forcing
function for step 1 of skills/meta/bespoke-composition.md.
Verified end-to-end on the compound-snowball proof render: positive case
status=pass, atelier checks clean; negative case (deliberate stock import
+ missing art_direction) returns status=fail with both findings named.
F7 — skills/meta/reviewer.md gets a "Composition Authoring Mode Review"
section that enforces the inversion at every stage:
- proposal: composition_mode must be a presented decision with both
options; hero briefs default to atelier; templated for hero requires
an explicit sanctioned reason.
- scene_plan/edit: any stock cut.type in an atelier piece is CRITICAL;
bespoke.{entry, composition_id, art_direction} required.
- compose: consumes final_review.checks.atelier from F6; mandates a
recorded distinctness review (the human taste call the tool can't
automate — its absence is itself CRITICAL).
- publish: blocks ship on any unresolved atelier finding.
Engine knowledge stays reusable; creative components don't.
Introduce a hand-authored, project-local Remotion render path that bypasses
the cut-schema and the stock scene-type registry, for hero/bespoke videos
that must look distinct from one another.
- video_compose: composition_mode="atelier" (or renderer_family="bespoke")
routes to _render_via_atelier, which renders a project-local entry under
remotion-composer/projects/<slug>/ with an optional per-project public_dir
(skips copying the bloated shared public/). No cut-schema, no stock registry.
- skills/meta/bespoke-composition.md: routing skill — art direction
(visual-style) -> motion principles (Disney 12) -> engine mechanics
(remotion-best-practices + stock components read only as a mechanics codex)
-> atelier render. Doctrine: reuse engine knowledge, never creative components.
- AGENT_GUIDE: "Composition Authoring Mode" (templated vs atelier); default
atelier for hero work; scene-type catalog reframed as a mechanics codex.
- animation-runtime-selector + INDEX: authoring-mode-first pointers.
- base_tool.run_command: decode subprocess output as UTF-8/replace (Windows
cp1252 crashed the reader thread on Remotion's Unicode progress output).
- .gitignore: remotion-composer/projects/ (throwaway bespoke compositions).
The compose target resolution was resolved from `profile` (and documented as
overridable via edit_decisions.metadata.compose_target) but the per-segment
scale/pad filter hardcoded 1920x1080. As a result, vertical profiles such as
`tiktok` / `youtube_shorts` / `instagram_reels` silently produced landscape
1920x1080 output instead of 1080x1920 — a silent dimension bug, no error raised.
- Use the resolved target width/height in the per-segment scale/pad filter.
- Implement the previously-stubbed `metadata.compose_target` extension point:
{"width", "height", "fit"} where fit="pad" (letterbox, default, unchanged
behavior) or fit="cover" (scale-to-fill + centre-crop, ideal for vertical).
- Default with no profile/target stays 1920x1080 (backward compatible).
Adds tests/tools/test_video_compose_vertical.py covering default landscape,
profile=tiktok vertical, and compose_target cover override.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
runway (estimate_cost/estimate_runtime/execute) defaulted to gen4_turbo and
higgsfield (execute) defaulted to kling_3.0, while both schemas advertise
seedance_2.0 as model.default. Omitting model under-quoted cost (runway 6x:
$0.25 vs $1.50) and silently generated a different model than advertised,
violating the Decision-Communication / cost-accuracy contract.
Root cause was a default duplicated across schema + 3 methods that drifted.
Collapse it to a single _DEFAULT_MODEL constant referenced everywhere.
Add tests/tools/test_provider_model_defaults.py to lock each tool's
estimate default to its schema default and guard the execute path.
