Commit Graph

70 Commits

Author SHA1 Message Date
Your Name
2beda014be fix(comfyui): pair wan22 t2v workflow with 14b-compatible vae 2026-07-04 19:45:47 +05:30
Calesthio
f4b8b90a24 Merge pull request #273 from calesthio/feat/backlot-living-storyboard
Release Backlot: the living storyboard for OpenMontage
2026-07-03 07:29:04 -07:00
Calesthio
0c202b507a Merge pull request #226 from 0xDevNinja/feat/publish-tier-export-bundle
feat(publish): add export_bundle, the first PUBLISH-tier tool
2026-07-02 15:06:03 -07:00
calesthio
280400d479 Ship Backlot living storyboard release hardening 2026-07-02 12:19:06 -07:00
0xDevNinja
8c9af320b6 fix(video_compose): forward remotion_timeout_ms through high-level render path
The timeout handling only took effect on a direct _remotion_render() call. The
high-level execute(operation='render') path goes through _render(), which builds
a fresh remotion_inputs dict (edit_decisions, output_path, profile) and dropped
remotion_timeout_ms — so callers of the documented operation='render' path never
got the timeout passed to the Remotion CLI. Forward it there.

Adds a test exercising _render() (not just _remotion_render()) to cover the
high-level forwarding path.

Refs #217
2026-07-02 11:59:53 +05:30
0xDevNinja
fbbe32a676 fix(video_compose): surface Remotion failures and add render timeout passthrough
The high-level Remotion render path hid the useful failure reason. run_command
runs with check=True + capture_output, so a non-zero exit raised
CalledProcessError whose str() is only 'returned non-zero exit status 1' — the
actual Remotion diagnostics in stderr were dropped. Catch CalledProcessError
and surface the stderr/stdout tail, and TimeoutExpired with an actionable hint.

Also add a creator-facing remotion_timeout_ms input, passed through as
Remotion's --timeout (headless-browser setup + delayRender). Slow browser
startup on restricted networks previously failed opaquely at the default 30s
with no way to raise it. The subprocess timeout is widened to match so
run_command does not kill Remotion before its own timeout fires.

Closes #217
2026-07-02 11:59:53 +05:30
calesthio
514d0faf37 backlot phase 0 review fixes: fail-closed gates, atomic checkpoint writes, event attribution hardening
- gate = manifest OR caller (stricter wins); unknown pipeline_type now
  raises instead of silently disabling enforcement; corrupt manifest logs
  and falls back; misleading diagnostic fixed
- write_checkpoint backfills pipeline_type from project.json marker so
  omitting the kwarg can't bypass gates
- checkpoint writes are atomic (temp + os.replace); history archiving is
  copy-based and best-effort (Windows open-file safe)
- manifest loads cached (load_pipeline_readonly); stage gate lookup moved
  to pipeline_loader.get_stage_human_approval_default; PROJECTS_DIR unified
  in lib/paths.py
- events: containment + root-normalization for explicit project dirs, no
  ghost-project mkdir, 0.0 cost preserved, nested-call depth tag,
  wrapper simplified
- documentary-montage edit-director gate footer (was missed); AGENT_GUIDE
  no longer claims edit/compose always auto-proceed
2026-07-01 23:24:11 -07:00
calesthio
722491d732 backlot phase 0: artifact contract + gate hardening
- init_project() writes project.json marker + canonical workspace layout
- write_checkpoint enforces approval gates: completed on a gated stage
  requires human_approved=True (GATE VIOLATION otherwise)
- superseded checkpoints archived to projects/<id>/history/ (stage
  versioning, gate audit trail, replay)
- BaseTool auto-instruments execute() -> projects/<id>/events.jsonl
  (start/finish/error, scene_id, cost) for the Backlot live board
- assets stage now gates (human_approval_default: true) in all manifests
- checkpoint-protocol + AGENT_GUIDE: manifest gate value is binding,
  awaiting_human + end-turn protocol, per-gate approval, canonical
  checkpoint location fixed to projects/<id>/
- gate reminder footer on all gating stage director skills
- /backlot command files for Claude Code, Codex, Cursor, Copilot
2026-07-01 23:08:51 -07:00
Remotely Human
fabc0e7088 Replace retired OpenAI image models with gpt-image-2 (DALL-E shut down 2026-05-12)
- openai_image: enum narrowed to ["gpt-image-2"]; DALL-E code path, sizes
  (1024x1792/1792x1024) and qualities (standard/hd) removed; estimate_cost
  updated to real GPT Image 2 pricing ($0.006/$0.053/$0.211 at 1024x1024)
- image_gen (deprecated but callable): default dall-e-3 -> gpt-image-2; drop
  response_format="b64_json", which GPT image models reject; update pricing
- docs/skills sweep: scrub retired-model references from README (en/zh),
  .env.example, docs/PROVIDERS.md (incl. pricing table + deprecation note),
  docs/ARCHITECTURE.md, AGENT_GUIDE.md, skills/creative/*, 14 pipeline
  director skills, and tests/qa/QA_PLAN.md

Source: https://developers.openai.com/api/docs/deprecations

proofread by fable 5 + codex
2026-07-02 11:36:31 +08:00
MoonCaves
07739952a8 fix openai_image default (gpt-image-1 is retired, 400s now) 2026-07-02 02:07:29 +08:00
Calesthio
1188e1bee9 Merge pull request #199 from An-idd/fix/video-compose-vertical-resolution
fix(video_compose): honor target resolution in FFmpeg compose (vertical/9:16)
2026-06-30 20:45:49 -07:00
Calesthio
7b8d59186d Merge pull request #223 from 0xDevNinja/feat/issue-168-music-library-preflight
feat(audio): add music_library tool so user tracks surface at preflight
2026-06-30 20:40:14 -07:00
0xDevNinja
9bcccce2db fix(publish): validate optional assets and default export inside project
Addresses review feedback on export_bundle:

- If subtitles_path or thumbnail_path is provided but the file is missing, the
  tool now fails with an explicit error instead of silently producing a package
  without that asset (which could ship an approved deliverable missing part of
  its content).
- Default export location now stays inside the project workspace: when the
  render lives at projects/<name>/renders/..., the bundle defaults to
  projects/<name>/exports/ (alongside artifacts/, assets/, renders/) rather than
  a repo-root exports/<name>/. export_dir remains an explicit override.

Tests cover both: missing optional asset errors, and the project-workspace
default path.
2026-06-30 12:05:23 +05:30
Calesthio
46ed550883 Merge pull request #229 from calesthio/codex/expressive-tts-governance
[codex] Add expressive TTS governance
2026-06-29 23:04:56 -07:00
Alastair Beal
85cf047c93 comfyui: route custom workflows through image/video selectors
The selector path previously hid the custom-workflow feature: video_selector
filtered tools on per-operation readiness (bundled WAN models) and both
selectors only chose ToolStatus.AVAILABLE providers, so comfyui_image/
comfyui_video — DEGRADED when bundled model metadata is missing — were
dropped even when the ComfyUI server was up and the caller supplied a full
workflow_json/workflow_path plus output_node.

- Add a custom-workflow readiness path to both selectors: when a custom
  workflow is supplied, eligibility is based on server availability (status
  != UNAVAILABLE) for any provider advertising supports.custom_workflow,
  not on bundled-model readiness. A custom workflow also restricts routing
  to custom-workflow-capable providers, since the graph JSON is ComfyUI
  specific.
- Expose workflow_json, workflow_path, output_node, workflow_name,
  workflow_model, and workflow_model_stack in both selector schemas so
  agents can discover the feature without bypassing the selectors.
- image_selector only forwards the workflow inputs to providers that
  declare them.
- Add contract tests for the new eligibility path and schema exposure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:38:18 +01:00
calesthio
5e4943a41b Add expressive TTS governance 2026-06-29 12:16:07 -07:00
calesthio
7c4bb08890 comfyui: satisfy provider contract review items 2026-06-29 19:09:22 +01:00
martimramos
4c62186c95 comfyui: drop music tool — ACE-Step node interface not standardized
Removed comfyui_music and its workflow. The ACE-Step model runs in
ComfyUI but the node class names differ across custom node packs
(AceStepModelLoader vs native TextEncodeAceStepAudio, etc.), so a
bundled workflow would break for most users.

Documented the reasoning in the plan doc and listed it as an open
question for future work. Users with ACE-Step working can still use
the workflow_json override on any tool.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-29 19:09:22 +01:00
martimramos
e3947e1f11 comfyui: add model discovery and actionable error messages
- Client queries ComfyUI /object_info to discover installed models
  (checkpoints, diffusion models, VAE, CLIP, LoRAs)
- Each tool declares its required models and checks them on execute()
- get_status() returns DEGRADED when server is up but models are missing
- Clear error messages tell the user exactly which models to download
- When COMFYUI_SERVER_URL is not set, error message tells the user to
  configure it in .env instead of silently failing on localhost:8188
- 8 new tests covering URL config, error messages, and model requirements

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-29 19:08:49 +01:00
martimramos
6ec2bbb090 comfyui: add native ComfyUI provider for image, video, and music generation
Adds three new BaseTool providers that delegate GPU work to a running
ComfyUI server via its REST API.  This avoids the need to install
PyTorch/diffusers directly, which is critical on hardware where the
ecosystem hasn't caught up (e.g. NVIDIA Blackwell / DGX Spark, aarch64
+ CUDA 13.0).

New files:
- tools/_comfyui/client.py — shared REST client (submit/poll/download)
- tools/_comfyui/workflows/ — 4 bundled workflow templates
- tools/graphics/comfyui_image.py — FLUX 2 Dev NVFP4 text-to-image
- tools/video/comfyui_video.py — WAN 2.2 14B t2v + i2v (4-step LightX2V)
- tools/audio/comfyui_music.py — ACE-Step 3.5B music generation
- tests/contracts/test_comfyui_tools.py — 41 contract tests
- docs/comfyui-adapter-plan.md — design document

Zero changes to existing tools, selectors, registry, or pipelines.
Tools are auto-discovered and selectors pick them up via capability match.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-29 19:08:49 +01:00
0xDevNinja
7b1eef49f2 feat(publish): add export_bundle, the first PUBLISH-tier tool
Every pipeline ends in a publish stage that produces a publish_log artifact,
but tools/publishers/ shipped empty (only __init__.py) — no tool backed the
PUBLISH tier, so the mechanical packaging (copy the render, write metadata
files, lay out the export directory, emit a schema-valid publish_log) had to be
hand-rolled by the agent every run.

Add a local, offline export bundler:
- capability 'publish', provider 'local', runtime 'local', deterministic, no cost
- takes the final video_path plus the SEO metadata the publish-director prepares
  (title, description, tags, hashtags, chapters, optional subtitles/thumbnail)
- writes exports/<project>/{video,metadata,thumbnails}/ matching the
  publish-director skill's documented layout
- returns a schema-valid publish_log (status: 'exported') in data, validated
  against schemas/artifacts/publish_log before returning so a bad entry fails
  here rather than at checkpoint

It does not upload — a networked publisher (e.g. YouTube) can be added later as
a separate provider under the same 'publish' capability.

Tests cover the contract, registry discovery, the export layout, chapter-time
formatting, the schema-valid publish_log, and the missing-video error path.
2026-06-29 15:40:59 +05:30
0xDevNinja
add955696d feat(audio): add music_library tool so user tracks surface at preflight
AGENT_GUIDE.md requires the music decision to be made at the proposal stage,
but the only check for the user's music_library/ folder lived in the
asset-director skills, which run later. A user could approve a creative
direction without ever being told a free, intentional music option was sitting
on disk (issue #168).

music_library/ was already referenced as a source_tool in asset artifacts but
had no backing tool. Add a small read-only tool that scans the library folder
(default <project root>/music_library, override via MUSIC_LIBRARY_DIR or a
library_dir input) and lists the audio tracks it finds, with best-effort
durations via ffprobe when present.

Because it inherits BaseTool, the registry auto-discovers it and it appears in
the preflight provider menu alongside music_gen and the stock music sources:
- AVAILABLE when the folder holds at least one audio track
- UNAVAILABLE otherwise, with install_instructions telling the user how to add
  tracks

So the user sees their music options before approving creative direction, with
no orchestration code changes. Read-only: no side effects, no cost.

Closes #168
2026-06-29 13:24:48 +05:30
calesthio
0f10381f2b feat(atelier): repo-root projects/, scaffolder, schemas (F1 + F3 + F5)
Closes the remaining three atelier audit gaps.

F1 — atelier projects live under repo-root projects/<slug>/ (the
established OpenMontage convention), not under remotion-composer/.
_render_via_atelier now auto-stages source files into
remotion-composer/projects/<slug>/ via mtime-skip copy when the entry
isn't already under the composer tree. Originally tried a directory
junction, but webpack walks up from the entry's REAL location to
resolve node_modules — junctions get dereferenced and module resolution
fails. Copy keeps source-of-truth under projects/, is invisible to the
user's project view, and is cheap (typical project is a few small .tsx
files). Output stays wherever output_path says. effective_entry is
returned in tool data for transparency.

F3 — scripts/scaffold_atelier_project.py generates the engine
boilerplate (index.tsx / Root.tsx / Composition.tsx / art-direction.md /
props template / README) ONLY. No creative content emitted — the
Composition.tsx is a deliberately blank black placeholder with prominent
TODOs. This removes the friction that nudged agents back to templated
mode while making it impossible to ship the scaffold unauthored (review
correctly flags black frames / silent audio as 'revise'). Verified
end-to-end: scaffold → render → review fires the right warnings.

F5 — schemas extended to know about atelier:
  - decision_log: 'composition_mode' added to category enum.
  - proposal_packet.production_plan: composition_mode + art_direction.
  - edit_decisions: composition_mode + bespoke{entry, composition_id,
    art_direction, props_path, public_dir, scale, crf, concurrency}.
  - final_review.checks: 'atelier' block (stock_reuse_detected,
    offending_imports[], art_direction_declared, art_direction).
  - final_review.recommended_action: 're_author' added.
  - final_review.transcript_comparison: allow null word_accuracy /
    transcript_matches_script (pre-existing latent bug surfaced here).
Both pass and fail shapes of the atelier render's final_review now
validate against the updated schema.
2026-06-27 11:15:36 -07:00
calesthio
5481226faf feat(atelier): post-render review hook + reviewer enforcement (F6 + F7)
Closes the two biggest gaps from the atelier audit: hero bespoke renders
were skipping every automated quality gate, and the templated→atelier
inversion was advisory rather than enforced.

F6 — _render_via_atelier now runs the full _run_final_review (technical
probe + visual spotcheck + audio spotcheck + duration drift + transcript
comparison) plus two atelier-specific doctrine checks:

  - stock_reuse_check: rglobs the bespoke project tree for imports from
    the stock creative registry (src/components, src/Explainer, etc.).
    Any hit fails the render with status="fail" and names the offending
    file — the doctrine bypass is now mechanically caught.
  - art_direction_check: requires edit_decisions.bespoke.art_direction.
    Missing → warning surfaced in final_review.issues_found; forcing
    function for step 1 of skills/meta/bespoke-composition.md.

Verified end-to-end on the compound-snowball proof render: positive case
status=pass, atelier checks clean; negative case (deliberate stock import
+ missing art_direction) returns status=fail with both findings named.

F7 — skills/meta/reviewer.md gets a "Composition Authoring Mode Review"
section that enforces the inversion at every stage:

  - proposal: composition_mode must be a presented decision with both
    options; hero briefs default to atelier; templated for hero requires
    an explicit sanctioned reason.
  - scene_plan/edit: any stock cut.type in an atelier piece is CRITICAL;
    bespoke.{entry, composition_id, art_direction} required.
  - compose: consumes final_review.checks.atelier from F6; mandates a
    recorded distinctness review (the human taste call the tool can't
    automate — its absence is itself CRITICAL).
  - publish: blocks ship on any unresolved atelier finding.

Engine knowledge stays reusable; creative components don't.
2026-06-27 11:01:00 -07:00
calesthio
578f0f1bb1 feat(compose): add atelier (bespoke) composition mode + doctrine
Introduce a hand-authored, project-local Remotion render path that bypasses
the cut-schema and the stock scene-type registry, for hero/bespoke videos
that must look distinct from one another.

- video_compose: composition_mode="atelier" (or renderer_family="bespoke")
  routes to _render_via_atelier, which renders a project-local entry under
  remotion-composer/projects/<slug>/ with an optional per-project public_dir
  (skips copying the bloated shared public/). No cut-schema, no stock registry.
- skills/meta/bespoke-composition.md: routing skill — art direction
  (visual-style) -> motion principles (Disney 12) -> engine mechanics
  (remotion-best-practices + stock components read only as a mechanics codex)
  -> atelier render. Doctrine: reuse engine knowledge, never creative components.
- AGENT_GUIDE: "Composition Authoring Mode" (templated vs atelier); default
  atelier for hero work; scene-type catalog reframed as a mechanics codex.
- animation-runtime-selector + INDEX: authoring-mode-first pointers.
- base_tool.run_command: decode subprocess output as UTF-8/replace (Windows
  cp1252 crashed the reader thread on Remotion's Unicode progress output).
- .gitignore: remotion-composer/projects/ (throwaway bespoke compositions).
2026-06-27 10:32:07 -07:00
An-idd
035df52b26 fix(video_compose): honor target resolution in FFmpeg compose (vertical/9:16)
The compose target resolution was resolved from `profile` (and documented as
overridable via edit_decisions.metadata.compose_target) but the per-segment
scale/pad filter hardcoded 1920x1080. As a result, vertical profiles such as
`tiktok` / `youtube_shorts` / `instagram_reels` silently produced landscape
1920x1080 output instead of 1080x1920 — a silent dimension bug, no error raised.

- Use the resolved target width/height in the per-segment scale/pad filter.
- Implement the previously-stubbed `metadata.compose_target` extension point:
  {"width", "height", "fit"} where fit="pad" (letterbox, default, unchanged
  behavior) or fit="cover" (scale-to-fill + centre-crop, ideal for vertical).
- Default with no profile/target stays 1920x1080 (backward compatible).

Adds tests/tools/test_video_compose_vertical.py covering default landscape,
profile=tiktok vertical, and compose_target cover override.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:56:49 +08:00
Calesthio
49a1e56825 Merge pull request #136 from vizionik25/feat/apple-silicon-mps-support
feat(gpu): Apple Silicon MPS (Metal) GPU Support
2026-06-26 14:29:55 -07:00
An-idd
93d96de8f1 fix(video): runway/higgsfield model defaults diverge from schema
runway (estimate_cost/estimate_runtime/execute) defaulted to gen4_turbo and
higgsfield (execute) defaulted to kling_3.0, while both schemas advertise
seedance_2.0 as model.default. Omitting model under-quoted cost (runway 6x:
$0.25 vs $1.50) and silently generated a different model than advertised,
violating the Decision-Communication / cost-accuracy contract.

Root cause was a default duplicated across schema + 3 methods that drifted.
Collapse it to a single _DEFAULT_MODEL constant referenced everywhere.

Add tests/tools/test_provider_model_defaults.py to lock each tool's
estimate default to its schema default and guard the execute path.
2026-06-26 15:30:22 +08:00
Calesthio
928311bf58 Merge pull request #191 from harshdadiya-wappnet/fix/skill-warnings-and-references
fix: resolve skill loading warnings and correct video-toolkit naming
2026-06-25 11:58:19 -07:00
Harsh Dadiya Wappnet
3656222356 fix: resolve skill loading warnings and correct video-toolkit naming 2026-06-25 19:00:47 +05:30
0xDevNinja
c5f6859a83 fix: use --props=<path> equals form for Remotion render
On Windows, passing --props and the JSON path as two separate CLI
arguments causes Remotion to mis-parse the value due to platform quote
escaping, failing with "neither valid JSON nor a file path to a valid
JSON file". Switch to the --props=<path> equals form, which Remotion
recommends for file paths and which works consistently across
platforms.

Fixes #172
2026-06-25 18:23:28 +05:30
Nik Nichols
211c3fcba4 feat(gpu): add device-aware routing for Apple Silicon MPS support
- Add get_torch_device() helper in _shared.py: cuda > mps > cpu
- Guard MPS detection for torch builds lacking torch.backends.mps
- Check both is_built() and is_available() for MPS
- Route load_diffusers_pipeline() to resolved device instead of hardcoded cuda
- Use float32 on CPU (float16 is emulated/unreliable), float16 on MPS, bfloat16 on CUDA
- Guard enable_model_cpu_offload() to CUDA-only; fall back to .to(device) on MPS
- Enable attention slicing for MPS memory safety
- Add inspect-based signature guard for device= arg on RealESRGANer/GFPGANer
- Update install_instructions on all LOCAL_GPU tools to mention MPS/Apple Silicon
2026-06-24 16:29:09 -05:00
Shubham
806f7ee1ac feat(google): support service-account auth for TTS + Imagen, fix false availability
Google's TTS and Imagen tools advertised service-account auth
(GOOGLE_APPLICATION_CREDENTIALS) but only ever authenticated with an API
key string, so users with a service-account JSON could not use either tool.
google_tts.get_status() also over-reported availability when the JSON was
set, then failed at execute() — a silent-availability bug.

Separately, both hand-rolled _load_dotenv parsers kept inline comments as
values, so after `cp .env.example .env` every keyed tool falsely reported
"available" with no real credentials.

Changes:
- Add tools/google_credentials.py: lazy google-auth Bearer-token helper.
- google_tts: authenticate via Cloud TTS Bearer token when only a service
  account is configured; make get_status() honest.
- google_imagen: route service-account auth to Vertex AI
  ({location}-aiplatform.googleapis.com) with project/location resolution,
  alongside the existing AI Studio API-key path.
- Fix both _load_dotenv parsers to strip inline comments (quote-aware).
- Add google-auth to requirements; document the new env vars in .env.example.
- .gitignore: never commit GCP service-account key files.

Verified locally with a real service account: TTS produced a valid MP3 and
Imagen produced a valid 1408x768 PNG via Vertex AI. Existing test suite
passes (2 unrelated pre-existing failures only).

Closes #131

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 17:53:10 +05:30
jiangjiahui5
3ee8548226 feat(tts): add Doubao Speech provider
Add Volcengine Doubao Speech 2.0 as a TTS provider for Mandarin narration with async timestamp metadata, provider documentation, and setup guidance.
2026-05-04 21:40:17 +08:00
calesthio
2b0801030c feat(character-animation): add local rigged character pipeline
New beta pipeline for reusable cartoon characters with SVG rigs, pose
libraries, action timelines, and Canvas/Remotion/HyperFrames rendering.

- pipeline_defs/character-animation.yaml: 11-stage manifest
- skills/pipelines/character-animation/: 11 stage director skills
- tools/character/: BaseTool implementations for char design, rigging,
  pose libraries, action timelines, previews, and QA
- schemas/artifacts/{character_design,rig_plan,pose_library,
  action_timeline,character_qa_report}.schema.json: canonical artifacts
- schemas/artifacts/scene_plan.schema.json: extended for character-led
  scenes
- .agents/skills/{canvas-procedural-animation,character-animation-qa,
  character-rigging,pose-library-design,svg-character-animation}/:
  Layer 3 vendor knowledge
- AGENT_GUIDE / PROJECT_CONTEXT / README / ARCHITECTURE / PROVIDERS:
  surface the new pipeline and its capability family
- tools/video/hyperframes_compose.py: SVG character rig support
- tests/contracts/test_character_animation_pipeline.py: contract tests
2026-04-28 08:11:02 -07:00
calesthio
386338c92b feat(remotion): upgrade cinematic TitleCard + add new scene components
- CinematicRenderer.tsx / cinematic/types.ts: TitleCard now supports
  ghosted video background, word-by-word stagger reveal with blur->sharp
  transition, proper newline handling, and growing accent lines. Title
  scenes accept backgroundSrc/backgroundTrimBeforeSeconds/
  backgroundTrimAfterSeconds plus a plate|overlay variant.
- Add CollageBurst, LyricOverlay, ScreenshotScene components.
- Explainer, Root, components/index.ts wiring updates.
- SCENE_TYPES.md reflects the new components.
- seedance-2-0 skill + seedance_video.py: small fixes.
2026-04-23 23:30:23 -07:00
itsuzef
099cde37b6 fix(seedance): correct upload_image_fal call name
Lines 219 and 227 imported upload_image_fal from _shared but called
the non-existent upload_image_to_fal, raising a NameError whenever
image_to_video or reference_to_video used local image paths.
2026-04-23 01:42:08 -04:00
calesthio
0efed7427c video-compose: catch TTS punctuation leaks in final review
Adds a transcript_comparison check to VideoCompose._run_final_review
that word-diffs the whisper/whisperx transcript against script.txt and
fails loudly when the TTS engine literally voiced punctuation tokens
(dot, dots, ellipsis, comma, dash, hyphen, period). Chirp3-HD did this
to ellipses in a production run and it slipped past review; now it
cannot.

Also corrects the "tiny background video" gotcha in
skills/core/hyperframes.md. After six renders of blaming HyperFrames
CSS, the real root cause was 640x360 Pexels sources combined with a
fit-and-pad pre-transform — HyperFrames was rendering the letterboxed
input faithfully. Gotcha now walks through the ffprobe diagnostic and
the scale-to-cover fix, and keeps the wrapper-div pattern for the
right reasons (aspect mismatch handling, not framework bug workaround).

Four regression tests cover the new check: punctuation-leak detection,
clean-audio false-positive guard, graceful skip when inputs missing,
and always-present transcript_comparison section.
2026-04-18 22:16:27 -07:00
calesthio
b4f7ec4eee hyperframes: add HTML/CSS/GSAP as a parallel composition runtime
Separates creative grammar (renderer_family) from technical engine
(render_runtime) so HyperFrames can stand alongside Remotion as a
first-class runtime instead of masquerading as a Remotion sub-case.
Locks runtime choice at proposal stage and enforces it end-to-end: the
schemas require it, video_compose routes by it, the reviewer fails
closed on silent swaps, and a parametrized contract test walks every
pipeline manifest to ensure each planning-stage skill explains the
conversation to the user. Adds hyperframes_compose (scaffold/lint/
validate/render/doctor/add_block), a playbook -> CSS style bridge, and
vendored HyperFrames Layer 3 skills from commit d291358, pinned via
PROVENANCE.md for future re-sync. Final_review now records
render_runtime_used and runtime_swap_detected so compose lies are
catchable after the fact.
2026-04-18 18:37:00 -07:00
calesthio
9e17263aa5 video-gen: prefer Seedance 2.0 on every gateway that supports it
Expands the Seedance 2.0 surface from fal.ai-only to every gateway the
OpenMontage toolset can reach, so users on Replicate / Runway /
Higgsfield automatically route cinematic briefs through Seedance 2.0
instead of the gateway's prior default.

- tools/video/seedance_replicate (new): REPLICATE_API_TOKEN adapter for
  bytedance/seedance-2.0 and bytedance/seedance-2.0-fast. Same provider
  family ("seedance"), quality_score=0.95, premium capability flags.
- tools/video/runway_video: add seedance_2.0 / seedance_2.0_fast to the
  model enum as the new default, flag native_audio + multi_shot +
  camera_direction + lip_sync in supports, point agent_skills at
  seedance-2-0 Layer 3 skill. Note Runway Unlimited/Enterprise + non-US
  gating in the model description.
- tools/video/higgsfield_video: same pattern. seedance_2.0 is now the
  default model. Cost estimates calibrated to Higgsfield credit pricing
  (~$0.50-$1.20 per 5s).
- tools/video/_shared: annotate HEYGEN_PROVIDERS seedance_pro/lite as
  Seedance 1.x (2.0 on HeyGen flows through Video Agent / Avatar Shots
  which need a separate wrapper — tracked as follow-up).
- .agents/skills/seedance-2-0: update provider matrix to show which
  gateways are wrapped (fal.ai, Replicate, Runway, Higgsfield) vs. which
  are still roadmap (HeyGen Video Agent, BytePlus, Freepik, aggregators).

The scoring engine already deduplicates by provider="seedance" so
whichever gateway the user has configured wins automatically. Agents
passing preferred_provider="seedance" to video_selector route to the
best available Seedance 2.0 path without naming a specific gateway.
2026-04-17 22:04:08 -07:00
calesthio
16791a3a80 video-gen: adopt Seedance 2.0 as preferred premium default
Seedance 2.0 is now routed as the top-ranked video generator whenever a
premium gateway is available. Touches the tool layer, scoring engine,
cinematic pipeline, and both skill layers so discovery works from every
entry point.

- tools/video/seedance_video: BETA stability, quality_score=0.95, add
  reference_to_video operation plus 9 img + 3 vid + 3 audio ceilings,
  fix pre-existing upload_image_fal import
- tools/base_tool: surface optional quality_score / success_rate /
  latency fields in get_info so the scorer can read them
- lib/scoring: fix reliability enum-vs-string bug that was pinning every
  available tool to 0.0, switch to overlap coefficient so rich best_for
  descriptions aren't penalized, add premium-cinematic feature bonus
- pipeline_defs/cinematic + cinematic asset-director: add pixabay_music
  and freesound_music, restore pixabay-first music default
- cinematic compose-director: mandatory Remotion preflight at stage entry
- New Layer 3 .agents/skills/seedance-2-0/SKILL.md (8-part prompt
  structure, multi-shot, lip-sync, reference-to-video, provider landscape)
- New Layer 2 skills/creative/prompting/seedance-prompting.md
- Update ai-video-gen, video-gen-prompting, AGENT_GUIDE, INDEX to flag
  Seedance 2.0 as the preferred premium default and make the skill
  discoverable from every routing path
2026-04-17 21:51:54 -07:00
calesthio
55c08acdd0 sources: add 11 stock source adapters, expand catalog from 5 to 16 providers
6 API-based adapters: Pixabay Video, Coverr, NARA (U.S. National Archives),
Library of Congress, Pond5 Public Domain, Videvo. 5 scraper-based adapters
for sites without APIs: ESA, NOAA, Mixkit, Dareful, JAXA. All follow the
StockSource protocol and are auto-discovered — no tool code changes needed.

Updated scene-director, asset-director, and idea-director skills with
source routing guidance so agents know which provider to use for which
content type (e.g. nara for historical, noaa for ocean, esa for space).

Also includes: grok_video capability updates (native audio, 1-15s duration,
new aspect ratios), seedance_video tool (Seedance 2.0 via fal.ai),
ProductReveal Remotion composition registration.
2026-04-13 12:34:58 -07:00
calesthio
0999eaddc7 docmontage: add direct_clip_search tool for fast provider-agnostic clip acquisition
Adds a lightweight alternative to the corpus_builder + clip_search pipeline
that skips CLIP embeddings, motion scores, and index files. Uses the same
StockSource adapter protocol so it works with all providers (Pexels,
Archive.org, NASA, Wikimedia, Unsplash). Asset-director skill now documents
both fast path (direct search) and standard path (corpus + CLIP retrieval).
Pipeline manifest updated to make corpus tools optional.
2026-04-12 14:23:14 -07:00
calesthio
cf3527fbbe docmontage: end-tag overlay default + Chirp 3 HD as default TTS voice
End-tag now composited over final body footage (ProRes 4444 with alpha)
instead of concatenated as a black card. Updated idea-director,
edit-director, compose-director, and pipeline manifest with overlay-first
instructions. Default Google TTS voice changed from Neural2-D to
Chirp3-HD-Orus across tool code and docs.
2026-04-11 21:38:36 -07:00
calesthio
a8d1ebdf6f docmontage: corpus builder hardening from P1 + P2 audit observations
All of these changes came out of running the P1 and P2 documentary-montage
audits end-to-end and watching specific things break. Grouping them into
one commit because they share a theme: making the corpus builder and its
stock source adapters robust enough that a real brief can produce a
real corpus without special-casing.

corpus_builder.py + new clip_cache.py + test_clip_cache.py
  Phase 1 of the shared-corpus architecture. Adds
  ~/.openmontage/clips_cache/ — a process-safe, LRU-evicted cache of
  downloaded clip files keyed by clip_id. Before each candidate download,
  corpus_builder asks the cache whether the bytes already exist on disk
  from a previous run; on a hit it hard-links (or copies on cross-drive)
  the blob into the caller's corpus dir and skips the network fetch. On
  a miss it downloads as usual and ingests the fresh file. Motivation:
  re-running the P1 audit after every tool fix was re-downloading gigs
  of archive.org footage that had already been fetched in the prior run.
  Cache faults never block the pipeline — they degrade gracefully to
  normal downloads. The cache bubbles counters into the corpus_builder
  return payload so the production report can show hit/miss/bytes-saved.
  Default 20 GB cap, overridable via OPENMONTAGE_CACHE_MAX_GB.
  Full test coverage: try_link, ingest, stats, LRU eviction, manifest
  persistence, lock behavior. 23 tests, tmp_path-scoped.

stock_sources/archive_org.py
  1. Three-strategy query cascade (phrase_prox_10 → distinctive_and →
     distinctive_or). Motivation: natural-language documentary queries
     against archive.org Solr were zeroing out — "1950s family watching
     television" returned 0 hits because Solr's default multi-term AND
     over-intersects. Walks strict to loose and returns the first
     non-empty strategy. Stop words, source hints ("prelinger",
     "archive", "footage"), and year tokens ("1950s") are excluded from
     the distinctive-token picks since they don't correlate with
     Prelinger title tokens.
  2. 150 MB per-rendition size cap. archive.org routinely hosts
     multi-hundred-megabyte h.264 masters and one 2 GB Prelinger item
     poisons corpus build wall-time and disk. Within a format bucket
     we now pick the largest rendition under the cap; if nothing fits
     we fall through to the next format rather than dropping the item.
  3. 180 s default max-duration ceiling when the caller hasn't set one
     — archive.org is the only source that routinely hosts feature-
     length material and a naive fan-out pulls them into corpora that
     only ever want a few seconds per clip.

stock_sources/wikimedia.py
  Parallel 3-strategy cascade (full → top2_or → single_best). Motivation:
  Commons CirrusSearch also defaults to AND across multi-word queries;
  our first P2 diagnostic pass returned 0 video results for 10/10
  queries. Same stop-word / source-hint / year-token stripping as
  archive_org so the two adapters stay symmetric.

test_stock_source_adapters.py
  Rewrote the wikimedia query-builder tests against the new cascade
  API. Added coverage for multi-word fallback + source-hint/year
  stripping.

video_compose.py
  Two small fixes for the Remotion renderer on Windows:
  1. Resolve output_path to absolute before invoking the CLI so the
     binary can write wherever the caller asked regardless of cwd.
  2. Pass cwd=composer_dir to run_command so npx can find the local
     Remotion binary under node_modules/.bin. Without this, Windows
     npx returns "could not determine executable to run" because it
     resolves .bin relative to the process cwd rather than the
     script's parent.
2026-04-11 00:46:06 -07:00
calesthio
de94d4dba3 Documentary Montage hardening plus governance fixes 2026-04-10 16:42:39 -07:00
calesthio
44baede67f Add documentary-montage pipeline for retrieval-first motion-clip montage
New end-to-end pipeline for building thematic documentary montages from
a locally-indexed corpus of free stock footage (Pexels, Archive.org,
NASA). The agent builds a project-local corpus, CLIP-ranks candidates
per scene slot, edits with motion-aware arc logic, and composes via
ffmpeg. No paid APIs required for the full path.

Pipeline definition and director skills:
- pipeline_defs/documentary-montage.yaml: 5-stage manifest
  (idea -> scene_plan -> assets -> edit -> compose)
- skills/pipelines/documentary-montage/: 6 director skills
  (executive-producer + idea/scene/asset/edit/compose directors)

Corpus and retrieval infrastructure:
- tools/video/corpus_builder.py: multi-source stock fan-out with
  resumable append-only corpus index
- tools/video/clip_search.py: CLIP ViT-B/32 retrieval —
  rank_for_slot, find_similar_set, diversify, stats
- tools/video/stock_sources/: base + pexels + archive_org + nasa
  adapters with a pluggable BaseStockSource contract
- lib/clip_embedder.py: CLIP wrapper
- lib/corpus.py: corpus schema, jsonl append/read, motion-score
  caching

video_compose fix rolled in because any concat-based pipeline depends
on it:
- Replace ambiguous -to with -t duration (was double-trimming cuts)
- Force re-encode + normalize to 1920x1080 @ 30fps (was keyframe-
  snapping with -c copy and breaking concat on mixed-source corpora)
- Add silent-audio anullsrc fallback for clips without an audio
  stream

README: add Documentary Montage row to the pipeline table and bump
the pipeline count from 11 to 12.
2026-04-10 16:04:28 -07:00
calesthio
a919fde450 Remove fal-first provider guidance 2026-04-08 13:04:32 -07:00
calesthio
4f682c8b0a Add Higgsfield provider and update Runway to v0.2.0
- New: Higgsfield video provider with multi-model routing (Kling 3.0, Veo 3.1, Sora 2, WAN 2.5, Soul Cinema) and Soul ID character consistency
- Updated: Runway provider with gen4_aleph and gen3a_turbo models, proper pixel-ratio mapping, probe_output, watermark param, RUNWAYML_API_SECRET env var support
- Docs: Updated provider counts (12→13), tool counts (51→52), added Higgsfield setup/pricing sections across README, AGENT_GUIDE, ARCHITECTURE, and PROVIDERS
2026-04-08 12:57:43 -07:00
calesthio
b1a078b282 Add screen capture tools with FFmpeg and Cap dual-provider system
New capture layer for the screen-demo pipeline: FFmpeg for quick CLI-driven
recording, Cap integration for polished recordings with webcam overlay and
cursor effects. Selector presents both options and routes based on availability.
2026-04-07 21:36:29 -07:00