diff --git a/.agents/skills/comfyui/SKILL.md b/.agents/skills/comfyui/SKILL.md index 64950870..ea89cc3e 100644 --- a/.agents/skills/comfyui/SKILL.md +++ b/.agents/skills/comfyui/SKILL.md @@ -1,17 +1,19 @@ --- name: comfyui -description: Use when working with ComfyUI workflows in OpenMontage, including comfyui_image/comfyui_video, custom workflow_json/workflow_path inputs, output_node selection, missing model setup, LoRAs, low-VRAM workflow choices, and community workflow imports. +description: Use when working with ComfyUI workflows in OpenMontage, including comfyui_image/comfyui_video/comfyui_music, custom workflow_json/workflow_path inputs, output_node selection, missing model setup, LoRAs, low-VRAM workflow choices, and community workflow imports. --- # ComfyUI Workflows in OpenMontage -Use this skill before calling `comfyui_image` or `comfyui_video`, and when converting a community ComfyUI workflow into an OpenMontage tool call. +Use this skill before calling `comfyui_image`, `comfyui_video`, or `comfyui_music`, and when converting a community ComfyUI workflow into an OpenMontage tool call. ## Server Contract - ComfyUI must be running before the tool can generate. The default server is `http://localhost:8188`; override it with `COMFYUI_SERVER_URL`. +- Running separate ComfyUI instances per capability (different GPU, different model set)? `COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL` / `COMFYUI_MUSIC_SERVER_URL` each override `COMFYUI_SERVER_URL` for that one tool only. Optional -- a single-server setup needs none of these. - Health and hardware status come from `GET /system_stats`. - Jobs are submitted to `POST /prompt`, completed outputs are read from `GET /history/{prompt_id}`, and artifact bytes are downloaded with `GET /view`. +- Long waits (video, music) prefer ComfyUI's websocket feed for immediate completion/error detection and transparently fall back to REST polling if `websocket-client` isn't installed. Either way, a timeout is recoverable: pass the error's `prompt_id` back in as `resume_prompt_id` to resume waiting on the same job instead of resubmitting it. - Export workflows with ComfyUI's API-format JSON, not the UI layout format. If a downloaded workflow will not submit, re-export it from ComfyUI with API format enabled. ## Choosing a Workflow @@ -52,4 +54,11 @@ Use this skill before calling `comfyui_image` or `comfyui_video`, and when conve - If the server is unavailable, surface the structured setup offer. Starting ComfyUI or setting `COMFYUI_SERVER_URL` is the first fix. - If models are missing, read `data.missing_models[]`; each item should include the file name, role, destination hint, and download URL when OpenMontage knows it. - If custom nodes are missing, ask the user to install them through ComfyUI Manager or the workflow author's documented install path, then restart ComfyUI. -- If a long render times out locally, check ComfyUI history before retrying from scratch; the server may still have completed the prompt. +- If a long render times out locally, check ComfyUI history before retrying from scratch; the server may still have completed the prompt -- or just call again with `resume_prompt_id` set to the `prompt_id` from the timeout error. + +## Music (`comfyui_music`) + +- Unlike `comfyui_image`/`comfyui_video`, there is **no bundled workflow**. ACE-Step's ComfyUI node interface isn't standardized across custom node packs (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.), so `workflow_json`/`workflow_path` + `output_node` are always required, not optional. +- `prompt` is provenance/logging only -- it is never injected into the workflow. Bake the actual tags/lyrics into the workflow JSON yourself before calling, the same way you would patch a custom image/video workflow. +- `output_node` should be the node that writes the final audio, typically ComfyUI's native `SaveAudio`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers). +- Provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow -- there's no bundled model stack to fall back on here. diff --git a/docs/comfyui-adapter-plan.md b/docs/comfyui-adapter-plan.md index c8eef2e3..c3d3884f 100644 --- a/docs/comfyui-adapter-plan.md +++ b/docs/comfyui-adapter-plan.md @@ -296,19 +296,44 @@ not promote ComfyUI for an operation whose bundled models are missing. --- -### `comfyui_music` -- Music Generation (not shipped) +### `comfyui_music` -- Music Generation (shipped, custom-workflow-only) -We explored adding a `comfyui_music` tool using the ACE-Step 3.5B model. -The model runs well in ComfyUI, but the ComfyUI node interface for -ACE-Step is not standardized -- there are multiple custom node packs with -different class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, -etc.). Shipping a workflow that only works with one specific custom node -pack would break for most users. +`tools/audio/comfyui_music.py`. `capability="music_generation"`, `provider="comfyui"`. +Ships with **no bundled workflow** -- the ACE-Step node-pack fragmentation +described below is real and unsolved, so instead of picking one pack and +breaking for everyone else, the tool always requires a caller-supplied +`workflow_json`/`workflow_path` + `output_node`, exactly like the image/video +tools' *optional* override path, just mandatory here. `prompt` is accepted +for provenance/logging only and is never injected into the workflow -- +tags/lyrics must already be baked into the graph before calling, same +convention as image/video custom workflows. -**Future path:** ACE-Step support should be revisited once OpenMontage decides -the music-generation routing shape and a portable ComfyUI audio workflow -contract. Current image/video workflow overrides are intentionally scoped to -image and video artifacts, not arbitrary audio workflows. +Originally not shipped because: the ComfyUI node interface for ACE-Step is +not standardized -- there are multiple custom node packs with different +class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.). +Shipping a workflow that only works with one specific custom node pack would +break for most users. The custom-workflow-only design sidesteps this +entirely: whichever node pack is installed, the caller exports it themselves. + +**Selector integration:** no dedicated `music_selector` exists in OpenMontage +(unlike `tts_selector`/`image_selector`/`video_selector`) -- music tools are +already routed directly via `registry.get_by_capability("music_generation")`, +and `comfyui_music` participates in that the same way `suno_music`/`music_gen` +do. `fallback_tools = ["suno_music", "music_gen"]`. + +**Audio artifact schema:** `ToolResult.data` follows the same shape as the +image/video tools (`provider`, `model`, `output`, `format`, `workflow_provenance`), +plus `duration_seconds` -- a best-effort `ffprobe` probe of the downloaded +file (`None` if `ffprobe` isn't on PATH), since a custom workflow gives no +other reliable way to know actual output duration ahead of time. + +**Workflow/output-node contract:** identical to image/video -- `output_node` +must be the ID of the node that writes the final artifact (typically ComfyUI's +native `SaveAudio` node). `ComfyUIClient.generate()`'s artifact extraction now +also checks the `"audio"` output key (previously only `"images"`/`"gifs"`), +which is what `SaveAudio` writes to in ComfyUI's `/history` response -- +this is the one part of the contract that *is* standardized regardless of +which ACE-Step loader pack sits upstream of it. --- @@ -373,15 +398,17 @@ COMFYUI_POLL_TIMEOUT=600 # max wait for image gen COMFYUI_VIDEO_TIMEOUT=900 # max wait for video gen ``` -**Multi-server (optional):** point `comfyui_image` and `comfyui_video` at -separate ComfyUI instances -- e.g. one GPU running FLUX 2, another running -WAN 2.2 -- by setting a per-capability override. Each takes priority over -`COMFYUI_SERVER_URL` for its own tool only; leave both unset and everything -still talks to the single shared server. +**Multi-server (optional):** point `comfyui_image`, `comfyui_video`, and +`comfyui_music` at separate ComfyUI instances -- e.g. one GPU running FLUX 2, +another running WAN 2.2, another running ACE-Step -- by setting a +per-capability override. Each takes priority over `COMFYUI_SERVER_URL` for +its own tool only; leave all three unset and everything talks to the single +shared server. ```bash COMFYUI_IMAGE_SERVER_URL=http://gpu-a:8188 COMFYUI_VIDEO_SERVER_URL=http://gpu-b:8188 +COMFYUI_MUSIC_SERVER_URL=http://gpu-c:8188 ``` **For Docker Compose setups** (ComfyUI in a container): @@ -496,20 +523,23 @@ pipeline definition, or any schema. `poll()` REST loop when it isn't installed or the connection fails — `resume_prompt_id` recovery behaves identically either way. -3. ~~**Multi-server:**~~ **Resolved.** `ComfyUIClient(capability="image"|"video")` +3. ~~**Multi-server:**~~ **Resolved.** `ComfyUIClient(capability="image"|"video"|"music")` resolves its server URL from a per-capability env var first - (`COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL`), then the shared - `COMFYUI_SERVER_URL`, then the `http://localhost:8188` default. `comfyui_image` - and `comfyui_video` pass their capability at construction, so image and video - generation can point at different ComfyUI instances (different GPUs, different - model sets) with zero code changes -- single-server setups need no extra - configuration since both env vars are optional. `client.capability`/ + (`COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL` / `COMFYUI_MUSIC_SERVER_URL`), + then the shared `COMFYUI_SERVER_URL`, then the `http://localhost:8188` default. + All three tools pass their capability at construction, so image, video, and + music generation can each point at different ComfyUI instances (different GPUs, + different model sets) with zero code changes -- single-server setups need no extra + configuration since all three env vars are optional. `client.capability`/ `client.is_default_url`/`client.unavailable_reason()` all account for the override, and `COMFYUI_SETUP_OFFER.per_capability_env_var_overrides` documents it for the setup-offer surfacing in `provider_menu()`. -4. **Music generation:** ACE-Step works in ComfyUI but OpenMontage needs a - dedicated music-generation routing contract before adding `comfyui_music`. - The follow-up should decide selector integration, audio artifact schemas, and - a portable workflow/output-node contract rather than treating music as a - hidden image/video workflow override. +4. ~~**Music generation:**~~ **Resolved -- shipped as custom-workflow-only.** + `comfyui_music` is a real tool now (not a hidden image/video override), routed + through the existing `registry.get_by_capability("music_generation")` path + like `suno_music`/`music_gen`. It has no bundled workflow -- the node-pack + fragmentation that originally blocked this is real, so the tool always + requires caller-supplied `workflow_json`/`workflow_path` + `output_node` + rather than betting on one pack. See the `comfyui_music` section above for + the artifact schema and workflow/output-node contract. diff --git a/tests/contracts/test_comfyui_tools.py b/tests/contracts/test_comfyui_tools.py index 44c451c0..27e32f7c 100644 --- a/tests/contracts/test_comfyui_tools.py +++ b/tests/contracts/test_comfyui_tools.py @@ -17,13 +17,14 @@ from tools.base_tool import ( ToolStatus, ToolTier, ) +from tools.audio.comfyui_music import ComfyUIMusic from tools.graphics.comfyui_image import ComfyUIImage from tools.graphics.image_selector import ImageSelector from tools.tool_registry import ToolRegistry from tools.video.video_selector import VideoSelector from tools.video.comfyui_video import ComfyUIVideo -TOOLS = [ComfyUIImage, ComfyUIVideo] +TOOLS = [ComfyUIImage, ComfyUIVideo, ComfyUIMusic] WORKFLOW_DIR = Path(__file__).resolve().parent.parent.parent / "tools" / "_comfyui" / "workflows" PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent @@ -323,6 +324,23 @@ class TestClientHelpers: ) assert paths == [tmp_path / "out.png"] + def test_generate_reads_audio_key_from_savaudio_node(self, monkeypatch, tmp_path): + """The native SaveAudio node writes outputs under "audio", not + "images"/"gifs" -- comfyui_music depends on this being handled.""" + from tools._comfyui.client import ComfyUIClient + + client = ComfyUIClient("http://comfy.test") + monkeypatch.setattr(client, "submit", lambda workflow: "p1") + monkeypatch.setattr(client, "poll", lambda prompt_id, **kwargs: { + "outputs": {"9": {"audio": [{ + "filename": "track.flac", "subfolder": "", "type": "output", + }]}} + }) + monkeypatch.setattr(client, "download", lambda filename, subfolder, dest, folder_type="output": Path(dest)) + + paths = client.generate({"9": {"inputs": {}}}, "9", tmp_path / "out.flac") + assert paths == [tmp_path / "out.flac"] + def test_is_default_url_when_env_not_set(self, monkeypatch): from tools._comfyui.client import ComfyUIClient monkeypatch.delenv("COMFYUI_SERVER_URL", raising=False) @@ -838,6 +856,142 @@ class TestCustomWorkflowContract: assert any(item["role"] == "vae" for item in provenance["model_stack"]) +class TestComfyUIMusic: + + def test_capability_and_provider(self): + tool = ComfyUIMusic() + assert tool.capability == "music_generation" + assert tool.provider == "comfyui" + + def test_requires_workflow_json_or_path(self): + tool = ComfyUIMusic() + tool._client.is_available = lambda: True + + result = tool.execute({"prompt": "ambient pad", "output_node": "9"}) + + assert result.success is False + assert "workflow_json" in result.error or "workflow_path" in result.error + + def test_requires_output_node(self): + tool = ComfyUIMusic() + tool._client.is_available = lambda: True + + result = tool.execute({ + "prompt": "ambient pad", + "workflow_json": json.dumps({"9": {"inputs": {}}}), + }) + + assert result.success is False + assert "output_node" in result.error + + def test_unavailable_server_reports_unavailable_reason(self): + tool = ComfyUIMusic() + tool._client.is_available = lambda: False + tool._client.unavailable_reason = lambda: "no server here" + + result = tool.execute({ + "prompt": "ambient pad", + "workflow_json": json.dumps({"9": {"inputs": {}}}), + "output_node": "9", + }) + + assert result.success is False + assert result.error == "no server here" + + def test_successful_generation_returns_provenance_and_duration(self, tmp_path, monkeypatch): + tool = ComfyUIMusic() + tool._client.is_available = lambda: True + + dest_file = tmp_path / "music.mp3" + + def fake_generate(workflow, output_node, dest, **kwargs): + Path(dest).write_bytes(b"fake-audio-bytes") + return [Path(dest)] + + tool._client.generate = fake_generate + monkeypatch.setattr("shutil.which", lambda name: None) # no ffprobe in test env + + result = tool.execute({ + "prompt": "upbeat synthwave", + "workflow_json": json.dumps({"9": {"inputs": {}}}), + "output_node": "9", + "output_path": str(dest_file), + "workflow_name": "my-ace-step-graph", + "workflow_model": "ace-step-v1-3.5b", + }) + + assert result.success is True + assert result.data["provider"] == "comfyui" + assert result.data["model"] == "ace-step-v1-3.5b" + assert result.data["output"] == str(dest_file) + assert result.data["format"] == "mp3" + assert result.data["duration_seconds"] is None # ffprobe unavailable + provenance = result.data["workflow_provenance"] + assert provenance["source"] == "user_supplied" + assert provenance["output_node"] == "9" + assert provenance["workflow_hash_sha256"] + + def test_timeout_surfaces_resumable_prompt_id(self, tmp_path): + from tools._comfyui.client import ComfyUIError + + tool = ComfyUIMusic() + tool._client.is_available = lambda: True + + def fake_generate(workflow, output_node, dest, **kwargs): + raise ComfyUIError("timed out", prompt_id="music-prompt-id") + + tool._client.generate = fake_generate + + result = tool.execute({ + "prompt": "ambient pad", + "workflow_json": json.dumps({"9": {"inputs": {}}}), + "output_node": "9", + "output_path": str(tmp_path / "music.mp3"), + }) + + assert result.success is False + assert result.data["prompt_id"] == "music-prompt-id" + assert "resume_prompt_id" in result.error + + def test_passes_timeout_and_resume_prompt_id_through(self, tmp_path): + tool = ComfyUIMusic() + tool._client.is_available = lambda: True + seen = {} + + def fake_generate(workflow, output_node, dest, **kwargs): + seen.update(kwargs) + return [Path(dest)] + + tool._client.generate = fake_generate + + tool.execute({ + "prompt": "ambient pad", + "workflow_json": json.dumps({"9": {"inputs": {}}}), + "output_node": "9", + "output_path": str(tmp_path / "music.mp3"), + "timeout_seconds": 3600, + "resume_prompt_id": "already-running-id", + }) + + assert seen["timeout"] == 3600 + assert seen["resume_prompt_id"] == "already-running-id" + + def test_registry_discovers_comfyui_music_under_music_generation(self): + registry = ToolRegistry() + tool = ComfyUIMusic() + registry.register(tool) + registry._discovered_packages.add("tools") + + by_capability = registry.get_by_capability("music_generation") + assert any(t.name == "comfyui_music" for t in by_capability) + + def test_uses_music_capability_env_var_for_multi_server(self, monkeypatch): + monkeypatch.delenv("COMFYUI_SERVER_URL", raising=False) + monkeypatch.setenv("COMFYUI_MUSIC_SERVER_URL", "http://music-gpu:8188") + tool = ComfyUIMusic() + assert tool._client.server_url == "http://music-gpu:8188" + + class TestComfyUISetupOffer: def test_provider_menu_summary_includes_structured_setup_offer(self): diff --git a/tools/_comfyui/client.py b/tools/_comfyui/client.py index 6632bfa6..e280bea6 100644 --- a/tools/_comfyui/client.py +++ b/tools/_comfyui/client.py @@ -404,8 +404,13 @@ class ComfyUIClient: outputs = entry.get("outputs", {}) node_output = outputs.get(output_node, {}) - # ComfyUI stores images and videos under the "images" key - items = node_output.get("images", []) or node_output.get("gifs", []) + # ComfyUI stores images/video frames under "images", legacy GIFs + # under "gifs", and the native SaveAudio node's output under "audio". + items = ( + node_output.get("images", []) + or node_output.get("gifs", []) + or node_output.get("audio", []) + ) if not items: raise ComfyUIError( f"No output artifacts on node {output_node}. " diff --git a/tools/_comfyui/metadata.py b/tools/_comfyui/metadata.py index 75707868..3c0ee13b 100644 --- a/tools/_comfyui/metadata.py +++ b/tools/_comfyui/metadata.py @@ -24,6 +24,7 @@ COMFYUI_SETUP_OFFER: dict[str, Any] = { "per_capability_env_var_overrides": { "comfyui_image": "COMFYUI_IMAGE_SERVER_URL", "comfyui_video": "COMFYUI_VIDEO_SERVER_URL", + "comfyui_music": "COMFYUI_MUSIC_SERVER_URL", }, } diff --git a/tools/audio/comfyui_music.py b/tools/audio/comfyui_music.py new file mode 100644 index 00000000..1d006f77 --- /dev/null +++ b/tools/audio/comfyui_music.py @@ -0,0 +1,300 @@ +"""ComfyUI music generation via a local or remote ComfyUI server. + +No bundled workflow: ACE-Step's ComfyUI node interface is not standardized +across custom node packs (``AceStepModelLoader`` vs native +``TextEncodeAceStepAudio``, etc.), so a hardcoded template would break for +most installs. This tool always runs a caller-supplied ``workflow_json`` or +``workflow_path`` -- the same override contract ``comfyui_image``/ +``comfyui_video`` offer as an alternative to their bundled workflow, just +mandatory here instead of optional. See the ``comfyui`` skill for how to +convert a community ACE-Step workflow into a call. +""" + +from __future__ import annotations + +import json +import shutil +import subprocess +import time +from pathlib import Path +from typing import Any + +from tools.base_tool import ( + BaseTool, + Determinism, + ExecutionMode, + ResourceProfile, + RetryPolicy, + ToolResult, + ToolRuntime, + ToolStability, + ToolStatus, + ToolTier, +) +from tools._comfyui.client import ComfyUIClient, ComfyUIError +from tools._comfyui.metadata import COMFYUI_SETUP_OFFER, workflow_hash + + +class ComfyUIMusic(BaseTool): + name = "comfyui_music" + version = "0.1.0" + tier = ToolTier.GENERATE + capability = "music_generation" + provider = "comfyui" + stability = ToolStability.EXPERIMENTAL + execution_mode = ExecutionMode.SYNC + determinism = Determinism.SEEDED + runtime = ToolRuntime.LOCAL_GPU + + dependencies = [] # checked at runtime via server health + setup_offer = COMFYUI_SETUP_OFFER + install_instructions = ( + "Start a ComfyUI server with ACE-Step installed (any node pack) and " + "set COMFYUI_SERVER_URL (default http://localhost:8188).\n" + "There is no bundled workflow for this tool -- export your ACE-Step " + "graph in API format and pass it as workflow_json/workflow_path.\n" + "Running a separate ComfyUI instance for music? Set " + "COMFYUI_MUSIC_SERVER_URL instead -- it takes priority over " + "COMFYUI_SERVER_URL for this tool only." + ) + agent_skills = ["comfyui"] + + capabilities = ["generate_background_music", "generate_song", "generate_instrumental"] + supports = { + "seed": True, + "custom_workflow": True, + "custom_output_node": True, + "offline": True, + } + best_for = [ + "local GPU music generation without API costs, using whatever ACE-Step node pack is installed", + "full control over sampling via custom ComfyUI workflows", + ] + not_good_for = [ + "setups without a running ComfyUI server", + "quick generation without first exporting/adapting an ACE-Step workflow", + "CPU-only machines", + ] + fallback_tools = ["suno_music", "music_gen"] + + input_schema = { + "type": "object", + "required": ["prompt", "output_node"], + "properties": { + "prompt": { + "type": "string", + "description": ( + "Description of the desired music, for provenance/logging only. " + "Not injected into the workflow -- bake the actual tags/lyrics " + "into workflow_json/workflow_path before calling." + ), + }, + "seed": {"type": "integer", "description": "Random if omitted"}, + "output_path": {"type": "string", "description": "Where to save the audio"}, + "workflow_json": { + "type": "string", + "description": "Full ComfyUI ACE-Step workflow JSON (API format). Required if workflow_path is omitted.", + }, + "workflow_path": { + "type": "string", + "description": "Path to a ComfyUI ACE-Step workflow JSON file. Required if workflow_json is omitted.", + }, + "output_node": { + "type": "string", + "description": "ComfyUI output node ID (e.g. the SaveAudio node) to download the artifact from.", + }, + "workflow_name": { + "type": "string", + "description": "Optional human-readable provenance label for the workflow.", + }, + "workflow_model": { + "type": "string", + "description": "Optional model/provenance label (e.g. 'ace-step-v1-3.5b').", + }, + "workflow_model_stack": { + "type": "array", + "description": ( + "Optional provenance metadata for workflow dependencies. " + "Items should include name, role, and node-pack origin when known." + ), + "items": {"type": "object"}, + }, + "timeout_seconds": { + "type": "integer", + "description": "How long to wait for the ComfyUI job before giving up. Default 1800s (30min).", + }, + "resume_prompt_id": { + "type": "string", + "description": "A prompt_id from a previous timed-out call. Skips resubmission and resumes waiting/downloading.", + }, + }, + } + + resource_profile = ResourceProfile( + cpu_cores=2, ram_mb=8000, vram_mb=8000, disk_mb=500, network_required=False, + ) + retry_policy = RetryPolicy(max_retries=1, retryable_errors=["timeout"]) + idempotency_key_fields = ["prompt", "seed", "workflow_json", "workflow_path", "output_node"] + side_effects = ["writes audio file to output_path"] + user_visible_verification = ["Listen to generated audio for mood, genre accuracy, and quality"] + + def __init__(self) -> None: + self._client = ComfyUIClient(capability="music") + self._last_progress_log = 0.0 + + def get_status(self) -> ToolStatus: + if not self._client.is_available(): + return ToolStatus.UNAVAILABLE + return ToolStatus.AVAILABLE + + def estimate_cost(self, inputs: dict[str, Any]) -> float: + return 0.0 + + def estimate_runtime(self, inputs: dict[str, Any]) -> float: + # Actual runtime depends entirely on the caller's custom workflow + # (steps, duration, sampler); this is a conservative flat estimate. + return 180.0 + + def get_info(self) -> dict[str, Any]: + info = super().get_info() + info["setup_offer"] = self.setup_offer + info["bundled_workflow"] = None + info["custom_workflow_required"] = True + return info + + def _log_progress(self, data: dict) -> None: + """Throttled progress line (see comfyui_video for rationale).""" + now = time.monotonic() + if now - self._last_progress_log < 10: + return + self._last_progress_log = now + value, max_value = data.get("value"), data.get("max") + if value is not None and max_value: + print(f"[comfyui_music] step {value}/{max_value}") + + def execute(self, inputs: dict[str, Any]) -> ToolResult: + if not (inputs.get("workflow_json") or inputs.get("workflow_path")): + return ToolResult( + success=False, + error=( + "comfyui_music requires workflow_json or workflow_path -- there " + "is no bundled default. ACE-Step's ComfyUI node interface isn't " + "standardized across custom node packs, so a hardcoded template " + "would break for most installs. Export the ACE-Step workflow " + "you actually have installed (API format) and pass it in." + ), + ) + + if not inputs.get("output_node"): + return ToolResult( + success=False, + error="output_node is required so OpenMontage knows which ComfyUI node to download the audio from.", + ) + + if not self._client.is_available(): + return ToolResult(success=False, error=self._client.unavailable_reason()) + + start = time.time() + seed = inputs.get("seed") or ComfyUIClient.random_seed() + output_path = Path(inputs.get("output_path", f"comfyui_music_{seed}.mp3")) + output_node = str(inputs["output_node"]) + + try: + workflow = self._load_custom_workflow(inputs) + provenance = self._workflow_provenance(inputs, output_node, workflow) + paths = self._client.generate( + workflow, + output_node=output_node, + dest=output_path, + timeout=inputs.get("timeout_seconds", 1800), + interval=10, + resume_prompt_id=inputs.get("resume_prompt_id"), + on_progress=self._log_progress, + ) + + except ComfyUIError as exc: + data = {"prompt_id": exc.prompt_id} if exc.prompt_id else {} + if exc.prompt_id: + error_msg = ( + f"{exc}\n\nThis job was NOT cancelled and is very likely still " + f"running server-side. To recover it without resubmitting, call " + f"execute() again with resume_prompt_id={exc.prompt_id!r} " + f"(and a longer timeout_seconds if it needs more time), or poll " + f"GET {{COMFYUI_SERVER_URL}}/history/{exc.prompt_id} directly." + ) + else: + error_msg = str(exc) + return ToolResult(success=False, error=error_msg, data=data) + except Exception as exc: + return ToolResult(success=False, error=f"ComfyUI music generation failed: {exc}") + + duration = self._probe_duration(paths[0]) + model_name = self._model_name(inputs) + return ToolResult( + success=True, + data={ + "provider": "comfyui", + "model": model_name, + "prompt": inputs["prompt"], + "duration_seconds": duration, + "output": str(paths[0]), + "format": paths[0].suffix.lstrip("."), + "workflow_provenance": provenance, + }, + artifacts=[str(p) for p in paths], + cost_usd=0.0, + duration_seconds=round(time.time() - start, 2), + seed=seed, + model=model_name, + ) + + @staticmethod + def _load_custom_workflow(inputs: dict[str, Any]) -> dict: + if inputs.get("workflow_json"): + return json.loads(inputs["workflow_json"]) + return ComfyUIClient.load_workflow(Path(inputs["workflow_path"])) + + @staticmethod + def _model_name(inputs: dict[str, Any]) -> str: + return ( + inputs.get("workflow_model") + or inputs.get("model") + or inputs.get("workflow_name") + or "custom-comfyui-workflow" + ) + + @staticmethod + def _workflow_provenance( + inputs: dict[str, Any], output_node: str, workflow: dict[str, Any] + ) -> dict[str, Any]: + stack = inputs.get("workflow_model_stack") + return { + "source": "user_supplied", + "workflow_name": inputs.get("workflow_name"), + "workflow_path": inputs.get("workflow_path"), + "model": inputs.get("workflow_model") or inputs.get("model"), + "workflow_hash_sha256": workflow_hash(workflow), + "model_stack": stack if isinstance(stack, list) else [], + "model_stack_source": "caller_supplied" if stack else "unknown_custom_workflow", + "output_node": output_node, + } + + @staticmethod + def _probe_duration(path: Path) -> float | None: + """Best-effort track duration via ffprobe; None if unavailable.""" + if shutil.which("ffprobe") is None: + return None + try: + out = subprocess.run( + [ + "ffprobe", "-v", "error", + "-show_entries", "format=duration", + "-of", "default=noprint_wrappers=1:nokey=1", + str(path), + ], + capture_output=True, text=True, timeout=15, check=True, + ) + value = out.stdout.strip() + return round(float(value), 2) if value else None + except (subprocess.SubprocessError, ValueError): + return None