diff --git a/AGENT_GUIDE.md b/AGENT_GUIDE.md index 74dd4c7e..3508302b 100644 --- a/AGENT_GUIDE.md +++ b/AGENT_GUIDE.md @@ -257,7 +257,7 @@ This returns every capability grouped by status — how many providers the user ``` YOUR CAPABILITIES - Video Generation: 0/12 configured + Video Generation: 0/13 configured Image Generation: 1/7 configured Text-to-Speech: 1/3 configured Music Generation: 1/1 configured @@ -272,7 +272,7 @@ For EACH capability with unavailable providers, read the `install_instructions` ``` QUICK SETUP OPTIONS (1-minute each — set an env var in .env) - Video Generation (0/12 -> unlock the biggest upgrade): + Video Generation (0/13 -> unlock the biggest upgrade): Each unavailable provider lists its own install_instructions. Read them from the provider_menu output and present grouped by env var. Example: if 3 tools need FAL_KEY, group them: "FAL_KEY unlocks 3 providers" diff --git a/README.md b/README.md index 7d88eccf..5c7a37da 100644 --- a/README.md +++ b/README.md @@ -273,7 +273,7 @@ Most AI video tools give you a single clip from a prompt. OpenMontage gives you Edit your own talking-head footage. Generate a fully animated explainer from scratch. Cut a 2-hour podcast into a dozen social clips. Translate and dub your content into 10 languages. Build a cinematic brand teaser from stock footage and AI-generated scenes. **If a production team can make it, OpenMontage can orchestrate it.** - **11 production pipelines** — explainers, talking heads, screen demos, cinematic trailers, animations, podcasts, localization, and more -- **51 production tools** — spanning video generation, image creation, text-to-speech, music, audio mixing, subtitles, enhancement, and analysis +- **52 production tools** — spanning video generation, image creation, text-to-speech, music, audio mixing, subtitles, enhancement, and analysis - **400+ agent skills** — production skills, pipeline directors, creative techniques, quality checklists, and deep technology knowledge packs that teach the agent how to use every tool like an expert - **Reference-driven creation** — paste a video you like and the agent turns it into a grounded, differentiated production plan instead of forcing you to invent the perfect prompt from scratch - **Live web research built in** — before writing a single word of script, the agent runs 15-25+ web searches across YouTube, Reddit, news sites, and academic sources to ground your video in real, current data @@ -370,14 +370,15 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to > **Full setup guide with pricing and free tiers:** [`docs/PROVIDERS.md`](docs/PROVIDERS.md)
-Video Generation — 12 providers +Video Generation — 13 providers | Provider | Type | Notes | |----------|------|-------| | **Kling** | Cloud API | High quality, fast | -| **Runway Gen-4** | Cloud API | Cinematic quality | +| **Runway Gen-4** | Cloud API | Cinematic quality, Gen-3 Alpha Turbo / Gen-4 Turbo / Gen-4 Aleph | | **Google Veo 3** | Cloud API | Long-form, cinematic. Via fal.ai or HeyGen. | | **Grok Imagine Video** | Cloud API | Strong reference-image video and xAI-native short-form generation | +| **Higgsfield** | Cloud API | Multi-model orchestrator with Soul ID for character consistency | | **MiniMax** | Cloud API | Cost-effective | | **HeyGen** | Cloud API | Multi-model gateway | | **WAN 2.1** | Local GPU | Free, 1.3B and 14B variants | diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 0ac2b5df..ce3b3f6c 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -51,7 +51,7 @@ OpenMontage/ │ ├── graphics/ # Image gen (FLUX, DALL-E, Recraft, local diffusion), stock, diagrams, code snippets, math animation │ ├── publishers/ # (Reserved) │ ├── subtitle/ # SRT/VTT generation from timestamps -│ └── video/ # 12 video gen providers, composition, stitching, trimming +│ └── video/ # 13 video gen providers, composition, stitching, trimming │ ├── pipeline_defs/ # 11 YAML pipeline manifests ├── schemas/ # JSON Schema definitions for validation @@ -162,7 +162,7 @@ Selectors route based on: user preference > availability > fallback order. They **Subtitle (1):** subtitle_gen -**Video (17):** grok_video, heygen_video, veo_video, kling_video, runway_video, minimax_video, wan_video, hunyuan_video, cogvideo_video, ltx_video_local, ltx_video_modal, pexels_video, pixabay_video, video_selector, video_compose (FFmpeg), video_stitch, video_trimmer +**Video (18):** grok_video, heygen_video, higgsfield_video, veo_video, kling_video, runway_video, minimax_video, wan_video, hunyuan_video, cogvideo_video, ltx_video_local, ltx_video_modal, pexels_video, pixabay_video, video_selector, video_compose (FFmpeg), video_stitch, video_trimmer --- @@ -384,7 +384,8 @@ All config is validated via Pydantic models in `lib/config_model.py`. | `PEXELS_API_KEY` | pexels_image, pexels_video | Stock media | | `PIXABAY_API_KEY` | pixabay_image, pixabay_video | Stock media | | `GOOGLE_API_KEY` | google_imagen, google_tts | Google Imagen images, Google Cloud TTS | -| `RUNWAY_API_KEY` | runway_video | Runway Gen-4 direct | +| `RUNWAY_API_KEY` | runway_video | Runway Gen-3/Gen-4 direct | +| `HIGGSFIELD_API_KEY` + `HIGGSFIELD_API_SECRET` | higgsfield_video | Higgsfield multi-model video | | `MODAL_LTX2_ENDPOINT_URL` | ltx_video_modal | Self-hosted LTX-2 | | `VIDEO_GEN_LOCAL_ENABLED` | local video tools | Enable local GPU generation | | `VIDEO_GEN_LOCAL_MODEL` | wan, hunyuan, ltx, cogvideo | Select local model | diff --git a/docs/PROVIDERS.md b/docs/PROVIDERS.md index 3e61988b..ed3ebce1 100644 --- a/docs/PROVIDERS.md +++ b/docs/PROVIDERS.md @@ -259,20 +259,19 @@ Google TTS offers 700+ voices across 50+ languages. Voice names follow the patte --- -### Runway — Gen-4 Video +### Runway — Gen-3/Gen-4 Video -> **Highest-rated AI video quality.** #1 on Elo rankings. Professional-grade video generation. +> **Highest-rated AI video quality.** #1 on Elo rankings. Professional-grade video generation with Gen-3 Alpha Turbo, Gen-4 Turbo, and Gen-4 Aleph models. **Tools unlocked:** `runway_video` **Env var:** `RUNWAY_API_KEY` #### Setup -1. Go to [app.runwayml.com/signup](https://app.runwayml.com/signup) and create an account +1. Go to [dev.runwayml.com](https://dev.runwayml.com/) and create a developer account 2. Subscribe to a paid plan (Standard or above — API requires subscription) -3. Go to **Settings > API Keys** at [app.runwayml.com/settings/api-keys](https://app.runwayml.com/settings/api-keys) -4. Click **Create API Key**, copy it -5. Add to `.env`: `RUNWAY_API_KEY=key_...` +3. Generate an API key from the developer portal +4. Add to `.env`: `RUNWAY_API_KEY=key_...` #### Pricing @@ -287,14 +286,56 @@ Google TTS offers 700+ voices across 50+ languages. Voice names follow the patte | Model | Price per second | |-------|-----------------| +| Gen-3 Alpha Turbo | ~$0.05 | | Gen-4 Turbo | ~$0.05 | -| Gen-4 | ~$0.10 | -| Gen-4.5 | ~$0.25 | +| Gen-4 Aleph | ~$0.15 | **Free tier:** 125 one-time credits (no monthly renewal). Enough for about 5 seconds of Gen-4 video. API access requires a paid subscription. --- +### Higgsfield — Multi-Model Video Orchestrator + +> **Multi-model video platform.** Routes to Kling 3.0, Veo 3.1, Sora 2, WAN 2.5, and proprietary Soul Cinema through a single API. Includes Soul ID for character consistency across clips. + +**Tools unlocked:** `higgsfield_video` +**Env vars:** `HIGGSFIELD_API_KEY` + `HIGGSFIELD_API_SECRET` (or combined `HIGGSFIELD_KEY=key:secret`) + +#### Setup + +1. Go to [cloud.higgsfield.ai](https://cloud.higgsfield.ai/) and create an account +2. Subscribe to a plan (Starter or above for API access) +3. Navigate to API Keys section at [cloud.higgsfield.ai/api-keys](https://cloud.higgsfield.ai/api-keys) +4. Generate an API key and secret +5. Add to `.env`: + ``` + HIGGSFIELD_API_KEY=your-api-key + HIGGSFIELD_API_SECRET=your-api-secret + ``` + +#### Pricing + +| Plan | Price | Notes | +|------|-------|-------| +| Free | $0 | Limited credits | +| Starter | $15/mo | Basic allocation | +| Plus | $34/mo | Mid-tier, ~33-56 Kling 3.0 clips | +| Ultra | $84/mo | High volume | + +**Per-generation costs (approximate, via credits):** + +| Model | Cost per clip | +|-------|--------------| +| Kling 3.0 | ~$0.10 (cheapest) | +| WAN 2.5 | ~$0.10 | +| Soul Cinema | ~$0.15 | +| Veo 3.1 | ~$0.50 | +| Sora 2 | ~$0.50 | + +**Free tier:** Limited credits on signup. No monthly renewal on free plan. + +--- + ### HeyGen — Avatar Video Gateway > **Multi-model video gateway.** Access VEO, Sora, Runway, Kling, and Seedance through a single API. @@ -603,6 +644,7 @@ These tools require only FFmpeg or Python packages — no GPU, no API key. | **OpenAI** | `OPENAI_API_KEY` | `openai_tts`, `openai_image` | Paid only | | **xAI** | `XAI_API_KEY` | `grok_image`, `grok_video` | Paid only | | **Runway** | `RUNWAY_API_KEY` | `runway_video` | Free trial + paid | +| **Higgsfield** | `HIGGSFIELD_API_KEY` + `HIGGSFIELD_API_SECRET` | `higgsfield_video` | Subscription ($15-84/mo) | | **HeyGen** | `HEYGEN_API_KEY` | `heygen_video` | Pay-as-you-go | | **Suno** | `SUNO_API_KEY` | `suno_music` | Pay-as-you-go | | **Local GPU** | `VIDEO_GEN_LOCAL_ENABLED` | `wan_video`, `hunyuan_video`, `cogvideo_video`, `ltx_video_local` | Free (GPU required) | @@ -618,7 +660,7 @@ How many providers cover each capability: | Capability | Cloud Providers | Local Providers | Free Options | |-----------|----------------|-----------------|--------------| | **Image Generation** | FLUX, Grok, Google Imagen, DALL-E 3, Recraft | Local Diffusion | Pexels, Pixabay (stock) | -| **Video Generation** | Grok, Kling, Runway, Veo, MiniMax, HeyGen | WAN, Hunyuan, CogVideo, LTX | Pexels, Pixabay (stock) | +| **Video Generation** | Grok, Kling, Runway, Veo, Higgsfield, MiniMax, HeyGen | WAN, Hunyuan, CogVideo, LTX | Pexels, Pixabay (stock) | | **Text-to-Speech** | ElevenLabs, Google TTS, OpenAI | Piper | Piper, Google free tier, ElevenLabs free tier | | **Music Generation** | ElevenLabs, Suno | — | ElevenLabs free tier | | **Post-Production** | — | FFmpeg (compose, stitch, trim, mix, enhance, grade) | All free | diff --git a/tools/video/higgsfield_video.py b/tools/video/higgsfield_video.py new file mode 100644 index 00000000..aaf83a2c --- /dev/null +++ b/tools/video/higgsfield_video.py @@ -0,0 +1,243 @@ +"""Higgsfield video generation via Higgsfield Cloud API. + +Multi-model orchestrator with proprietary Soul model for character-consistent, +photorealistic video generation. Routes to Kling, Veo, Sora, and WAN under the hood. +""" + +from __future__ import annotations + +import os +import time +from pathlib import Path +from typing import Any + +from tools.base_tool import ( + BaseTool, + Determinism, + ExecutionMode, + ResourceProfile, + RetryPolicy, + ToolResult, + ToolRuntime, + ToolStability, + ToolStatus, + ToolTier, +) + + +class HiggsFieldVideo(BaseTool): + name = "higgsfield_video" + version = "0.1.0" + tier = ToolTier.GENERATE + capability = "video_generation" + provider = "higgsfield" + stability = ToolStability.EXPERIMENTAL + execution_mode = ExecutionMode.SYNC + determinism = Determinism.STOCHASTIC + runtime = ToolRuntime.API + + dependencies = [] + install_instructions = ( + "Set HIGGSFIELD_API_KEY and HIGGSFIELD_API_SECRET for your Higgsfield Cloud credentials.\n" + " Get them at https://cloud.higgsfield.ai/api-keys\n" + " Alternatively, set HIGGSFIELD_KEY as a combined key:secret value." + ) + agent_skills = ["ai-video-gen"] + + capabilities = ["text_to_video", "image_to_video"] + supports = { + "text_to_video": True, + "image_to_video": True, + "character_consistency": True, + "multi_model_routing": True, + } + best_for = [ + "character-consistent video generation (Soul ID)", + "multi-model access through a single API", + "photorealistic and fashion-aware content", + ] + not_good_for = ["offline generation", "fine-grained model control", "budget projects without subscription"] + fallback_tools = ["kling_video", "veo_video", "minimax_video"] + + input_schema = { + "type": "object", + "required": ["prompt"], + "properties": { + "prompt": {"type": "string"}, + "operation": { + "type": "string", + "enum": ["text_to_video", "image_to_video"], + "default": "text_to_video", + }, + "model": { + "type": "string", + "enum": [ + "kling_3.0", + "veo_3.1", + "sora_2", + "wan_2.5", + "soul_cinema", + ], + "default": "kling_3.0", + "description": "Underlying model to use for generation", + }, + "duration": { + "type": "string", + "enum": ["5", "10", "15"], + "default": "5", + "description": "Duration in seconds (availability varies by model)", + }, + "aspect_ratio": { + "type": "string", + "enum": ["16:9", "9:16", "1:1", "21:9"], + "default": "16:9", + }, + "image_url": {"type": "string", "description": "Reference image URL for image_to_video"}, + "output_path": {"type": "string"}, + }, + } + + resource_profile = ResourceProfile( + cpu_cores=1, ram_mb=512, vram_mb=0, disk_mb=500, network_required=True + ) + retry_policy = RetryPolicy(max_retries=2, retryable_errors=["rate_limit", "timeout"]) + idempotency_key_fields = ["prompt", "model", "operation", "duration"] + side_effects = ["writes video file to output_path", "calls Higgsfield Cloud API"] + user_visible_verification = ["Watch generated clip for motion coherence and visual quality"] + + def _get_credentials(self) -> tuple[str, str] | None: + """Return (api_key, api_secret) or None if not configured.""" + combined = os.environ.get("HIGGSFIELD_KEY") + if combined and ":" in combined: + key, secret = combined.split(":", 1) + return key, secret + key = os.environ.get("HIGGSFIELD_API_KEY") + secret = os.environ.get("HIGGSFIELD_API_SECRET") + if key and secret: + return key, secret + return None + + def get_status(self) -> ToolStatus: + if self._get_credentials(): + return ToolStatus.AVAILABLE + return ToolStatus.UNAVAILABLE + + def estimate_cost(self, inputs: dict[str, Any]) -> float: + model = inputs.get("model", "kling_3.0") + duration = int(inputs.get("duration", "5")) + # Approximate per-clip costs based on Higgsfield credit pricing + base_costs = { + "kling_3.0": 0.10, + "wan_2.5": 0.10, + "veo_3.1": 0.50, + "sora_2": 0.50, + "soul_cinema": 0.15, + } + base = base_costs.get(model, 0.15) + return base * (duration / 5) + + def estimate_runtime(self, inputs: dict[str, Any]) -> float: + model = inputs.get("model", "kling_3.0") + if model in ("veo_3.1", "sora_2"): + return 120.0 + return 60.0 + + def execute(self, inputs: dict[str, Any]) -> ToolResult: + creds = self._get_credentials() + if not creds: + return ToolResult( + success=False, + error="Higgsfield credentials not set. " + self.install_instructions, + ) + + import requests + + api_key, api_secret = creds + start = time.time() + operation = inputs.get("operation", "text_to_video") + model = inputs.get("model", "kling_3.0") + + payload: dict[str, Any] = { + "prompt": inputs["prompt"], + "model": model, + "task": operation.replace("_", "-"), + } + if inputs.get("duration"): + payload["duration"] = int(inputs["duration"]) + if inputs.get("aspect_ratio"): + payload["aspect_ratio"] = inputs["aspect_ratio"] + if operation == "image_to_video" and inputs.get("image_url"): + payload["image_url"] = inputs["image_url"] + + headers = { + "Authorization": f"Bearer {api_key}", + "X-API-Secret": api_secret, + "Content-Type": "application/json", + } + + try: + # Submit generation request + submit_resp = requests.post( + "https://platform.higgsfield.ai/v1/generations", + headers=headers, + json=payload, + timeout=30, + ) + submit_resp.raise_for_status() + gen_data = submit_resp.json() + generation_id = gen_data["id"] + status_url = gen_data.get("status_url", f"https://platform.higgsfield.ai/v1/generations/{generation_id}") + + # Poll for completion + video_url = None + for _ in range(72): # max ~6 minutes + time.sleep(5) + poll_resp = requests.get(status_url, headers=headers, timeout=15) + poll_resp.raise_for_status() + poll_data = poll_resp.json() + status = poll_data.get("status", "Unknown") + + if status in ("Completed", "COMPLETED"): + video_url = poll_data.get("output_url") or poll_data.get("url") + break + if status in ("Failed", "FAILED", "NSFW", "Cancelled", "CANCELLED"): + return ToolResult( + success=False, + error=f"Higgsfield generation {status}: {poll_data.get('error', 'unknown')}", + ) + + if not video_url: + return ToolResult(success=False, error="Higgsfield generation timed out.") + + # Download video + video_response = requests.get(video_url, timeout=120) + video_response.raise_for_status() + + output_path = Path(inputs.get("output_path", "higgsfield_output.mp4")) + output_path.parent.mkdir(parents=True, exist_ok=True) + output_path.write_bytes(video_response.content) + + except Exception as e: + return ToolResult(success=False, error=f"Higgsfield video generation failed: {e}") + + from tools.video._shared import probe_output + + probed = probe_output(output_path) + return ToolResult( + success=True, + data={ + "provider": "higgsfield", + "model": model, + "prompt": inputs["prompt"], + "operation": operation, + "aspect_ratio": inputs.get("aspect_ratio", "16:9"), + "output": str(output_path), + "output_path": str(output_path), + "format": "mp4", + **probed, + }, + artifacts=[str(output_path)], + cost_usd=self.estimate_cost(inputs), + duration_seconds=round(time.time() - start, 2), + model=model, + ) diff --git a/tools/video/runway_video.py b/tools/video/runway_video.py index 8ed84460..6de0eef3 100644 --- a/tools/video/runway_video.py +++ b/tools/video/runway_video.py @@ -1,6 +1,7 @@ """Runway Gen-4 video generation via Runway API. Highest Elo-rated video generation model — professional quality and control. +Supports Gen-3 Alpha Turbo, Gen-4 Turbo, and Gen-4 Aleph (highest fidelity). """ from __future__ import annotations @@ -23,22 +24,40 @@ from tools.base_tool import ( ToolTier, ) +_RATIO_MAP = { + "16:9": "1280:720", + "9:16": "720:1280", + "1:1": "720:720", +} + +_COST_PER_SECOND = { + "gen3a_turbo": 0.05, + "gen4_turbo": 0.05, + "gen4_aleph": 0.15, +} + +_RUNTIME_SECONDS = { + "gen3a_turbo": 25.0, + "gen4_turbo": 30.0, + "gen4_aleph": 60.0, +} + class RunwayVideo(BaseTool): name = "runway_video" - version = "0.1.0" + version = "0.2.0" tier = ToolTier.GENERATE capability = "video_generation" provider = "runway" - stability = ToolStability.EXPERIMENTAL + stability = ToolStability.BETA execution_mode = ExecutionMode.SYNC determinism = Determinism.STOCHASTIC runtime = ToolRuntime.API dependencies = [] install_instructions = ( - "Set RUNWAY_API_KEY to your Runway API key.\n" - " Get one at https://app.runwayml.com/settings/api-keys" + "Set RUNWAY_API_KEY to your Runway API secret.\n" + " Get one at https://dev.runwayml.com/" ) agent_skills = ["ai-video-gen"] @@ -68,8 +87,9 @@ class RunwayVideo(BaseTool): }, "model": { "type": "string", - "enum": ["gen4_turbo", "gen4"], + "enum": ["gen4_turbo", "gen4_aleph", "gen3a_turbo"], "default": "gen4_turbo", + "description": "gen4_aleph is highest fidelity, gen4_turbo is balanced, gen3a_turbo is cheapest", }, "duration": { "type": "integer", @@ -82,6 +102,11 @@ class RunwayVideo(BaseTool): "enum": ["16:9", "9:16", "1:1"], "default": "16:9", }, + "watermark": { + "type": "boolean", + "default": False, + "description": "Include Runway watermark on output", + }, "image_url": {"type": "string", "description": "Reference image URL for image_to_video"}, "output_path": {"type": "string"}, }, @@ -90,29 +115,30 @@ class RunwayVideo(BaseTool): resource_profile = ResourceProfile( cpu_cores=1, ram_mb=512, vram_mb=0, disk_mb=500, network_required=True ) - retry_policy = RetryPolicy(max_retries=2, retryable_errors=["rate_limit", "timeout"]) + retry_policy = RetryPolicy(max_retries=2, retryable_errors=["rate_limit", "timeout", "THROTTLED"]) idempotency_key_fields = ["prompt", "model", "operation", "duration"] side_effects = ["writes video file to output_path", "calls Runway API"] user_visible_verification = ["Watch generated clip for visual quality and motion coherence"] def get_status(self) -> ToolStatus: - if os.environ.get("RUNWAY_API_KEY"): + if os.environ.get("RUNWAY_API_KEY") or os.environ.get("RUNWAYML_API_SECRET"): return ToolStatus.AVAILABLE return ToolStatus.UNAVAILABLE + def _get_api_key(self) -> str | None: + return os.environ.get("RUNWAY_API_KEY") or os.environ.get("RUNWAYML_API_SECRET") + def estimate_cost(self, inputs: dict[str, Any]) -> float: + model = inputs.get("model", "gen4_turbo") duration = inputs.get("duration", 5) - # Runway charges per second of generated video - return 0.05 * duration # ~$0.25 for 5s, ~$0.50 for 10s + return _COST_PER_SECOND.get(model, 0.05) * duration def estimate_runtime(self, inputs: dict[str, Any]) -> float: model = inputs.get("model", "gen4_turbo") - if "turbo" in model: - return 30.0 - return 60.0 + return _RUNTIME_SECONDS.get(model, 30.0) def execute(self, inputs: dict[str, Any]) -> ToolResult: - api_key = os.environ.get("RUNWAY_API_KEY") + api_key = self._get_api_key() if not api_key: return ToolResult( success=False, @@ -124,17 +150,26 @@ class RunwayVideo(BaseTool): start = time.time() model = inputs.get("model", "gen4_turbo") operation = inputs.get("operation", "text_to_video") + ratio_friendly = inputs.get("ratio", "16:9") + ratio_pixels = _RATIO_MAP.get(ratio_friendly, "1280:720") - # Runway API v1 — submit task task_payload: dict[str, Any] = { "model": model, "promptText": inputs["prompt"], "duration": inputs.get("duration", 5), - "ratio": inputs.get("ratio", "16:9"), + "ratio": ratio_pixels, + "watermark": inputs.get("watermark", False), } if operation == "image_to_video" and inputs.get("image_url"): task_payload["promptImage"] = inputs["image_url"] + # Choose endpoint based on operation + endpoint = ( + "https://api.dev.runwayml.com/v1/image_to_video" + if operation == "image_to_video" + else "https://api.dev.runwayml.com/v1/text_to_video" + ) + headers = { "Authorization": f"Bearer {api_key}", "Content-Type": "application/json", @@ -144,8 +179,7 @@ class RunwayVideo(BaseTool): try: # Submit generation task submit_response = requests.post( - "https://api.dev.runwayml.com/v1/image_to_video" if operation == "image_to_video" - else "https://api.dev.runwayml.com/v1/text_to_video", + endpoint, headers=headers, json=task_payload, timeout=30, @@ -153,9 +187,9 @@ class RunwayVideo(BaseTool): submit_response.raise_for_status() task_id = submit_response.json()["id"] - # Poll for completion + # Poll for completion (max ~5 minutes) video_url = None - for _ in range(60): # max 5 minutes + for _ in range(60): time.sleep(5) poll_response = requests.get( f"https://api.dev.runwayml.com/v1/tasks/{task_id}", @@ -164,20 +198,23 @@ class RunwayVideo(BaseTool): ) poll_response.raise_for_status() task_data = poll_response.json() + status = task_data["status"] - if task_data["status"] == "SUCCEEDED": + if status == "SUCCEEDED": video_url = task_data["output"][0] break - if task_data["status"] == "FAILED": + if status == "FAILED": + failure_code = task_data.get("failureCode", "unknown") return ToolResult( success=False, - error=f"Runway generation failed: {task_data.get('failure', 'unknown error')}", + error=f"Runway generation failed ({failure_code}): {task_data.get('failure', 'unknown error')}", ) + # PENDING, THROTTLED, RUNNING — keep polling if not video_url: - return ToolResult(success=False, error="Runway generation timed out.") + return ToolResult(success=False, error="Runway generation timed out after 5 minutes.") - # Download video + # Download video — URLs are ephemeral (expire in 24-48h) video_response = requests.get(video_url, timeout=120) video_response.raise_for_status() @@ -188,14 +225,22 @@ class RunwayVideo(BaseTool): except Exception as e: return ToolResult(success=False, error=f"Runway video generation failed: {e}") + from tools.video._shared import probe_output + + probed = probe_output(output_path) return ToolResult( success=True, data={ "provider": "runway", "model": model, "prompt": inputs["prompt"], + "operation": operation, + "ratio": ratio_friendly, "output": str(output_path), + "output_path": str(output_path), "task_id": task_id, + "format": "mp4", + **probed, }, artifacts=[str(output_path)], cost_usd=self.estimate_cost(inputs),