zlei6/OpenMontage
1
0
Fork 0
mirror of https://github.com/calesthio/OpenMontage.git synced 2026-08-16 13:36:26 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: ship comfyui_music as a custom-workflow-only ACE-Step tool

Browse Source 
Resolves the "music generation" open question from the adapter plan.
Unlike comfyui_image/comfyui_video there is no bundled workflow: ACE-Step's
ComfyUI node interface isn't standardized across custom node packs
(AceStepModelLoader vs native TextEncodeAceStepAudio, etc.), so instead of
picking one pack and breaking for everyone else, comfyui_music always
requires a caller-supplied workflow_json/workflow_path + output_node --
the same override contract image/video offer as an alternative, just
mandatory here. prompt is provenance-only, never injected into the graph.

Routed through the existing registry.get_by_capability("music_generation")
path alongside suno_music/music_gen -- no dedicated selector needed.
ComfyUIClient.generate() now also reads the "audio" output key (what
ComfyUI's native SaveAudio node writes), and gets timeout/resume/websocket-
wait/multi-server support for free via the shared client. Duration is a
best-effort ffprobe probe of the downloaded file since a custom workflow
gives no other way to know it ahead of time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Ntsako
2026-08-06 14:19:05 +02:00
parent 2f114682e8
commit 172acca6ea
6 changed files with 533 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
.agents/skills/comfyui/SKILL.md
View File

@@ -1,17 +1,19 @@
---
name: comfyui
description: Use when working with ComfyUI workflows in OpenMontage, including comfyui_image/comfyui_video, custom workflow_json/workflow_path inputs, output_node selection, missing model setup, LoRAs, low-VRAM workflow choices, and community workflow imports.
description: Use when working with ComfyUI workflows in OpenMontage, including comfyui_image/comfyui_video/comfyui_music, custom workflow_json/workflow_path inputs, output_node selection, missing model setup, LoRAs, low-VRAM workflow choices, and community workflow imports.
---
# ComfyUI Workflows in OpenMontage
Use this skill before calling `comfyui_image` or `comfyui_video`, and when converting a community ComfyUI workflow into an OpenMontage tool call.
Use this skill before calling `comfyui_image`, `comfyui_video`, or `comfyui_music`, and when converting a community ComfyUI workflow into an OpenMontage tool call.
## Server Contract
- ComfyUI must be running before the tool can generate. The default server is `http://localhost:8188`; override it with `COMFYUI_SERVER_URL`.
- Running separate ComfyUI instances per capability (different GPU, different model set)? `COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL` / `COMFYUI_MUSIC_SERVER_URL` each override `COMFYUI_SERVER_URL` for that one tool only. Optional -- a single-server setup needs none of these.
- Health and hardware status come from `GET /system_stats`.
- Jobs are submitted to `POST /prompt`, completed outputs are read from `GET /history/{prompt_id}`, and artifact bytes are downloaded with `GET /view`.
- Long waits (video, music) prefer ComfyUI's websocket feed for immediate completion/error detection and transparently fall back to REST polling if `websocket-client` isn't installed. Either way, a timeout is recoverable: pass the error's `prompt_id` back in as `resume_prompt_id` to resume waiting on the same job instead of resubmitting it.
- Export workflows with ComfyUI's API-format JSON, not the UI layout format. If a downloaded workflow will not submit, re-export it from ComfyUI with API format enabled.
## Choosing a Workflow
@@ -52,4 +54,11 @@ Use this skill before calling `comfyui_image` or `comfyui_video`, and when conve
- If the server is unavailable, surface the structured setup offer. Starting ComfyUI or setting `COMFYUI_SERVER_URL` is the first fix.
- If models are missing, read `data.missing_models[]`; each item should include the file name, role, destination hint, and download URL when OpenMontage knows it.
- If custom nodes are missing, ask the user to install them through ComfyUI Manager or the workflow author's documented install path, then restart ComfyUI.
- If a long render times out locally, check ComfyUI history before retrying from scratch; the server may still have completed the prompt.
- If a long render times out locally, check ComfyUI history before retrying from scratch; the server may still have completed the prompt -- or just call again with `resume_prompt_id` set to the `prompt_id` from the timeout error.
## Music (`comfyui_music`)
- Unlike `comfyui_image`/`comfyui_video`, there is **no bundled workflow**. ACE-Step's ComfyUI node interface isn't standardized across custom node packs (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.), so `workflow_json`/`workflow_path` + `output_node` are always required, not optional.
- `prompt` is provenance/logging only -- it is never injected into the workflow. Bake the actual tags/lyrics into the workflow JSON yourself before calling, the same way you would patch a custom image/video workflow.
- `output_node` should be the node that writes the final audio, typically ComfyUI's native `SaveAudio`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers).
- Provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow -- there's no bundled model stack to fall back on here.

86
docs/comfyui-adapter-plan.md
View File

@@ -296,19 +296,44 @@ not promote ComfyUI for an operation whose bundled models are missing.
---
### `comfyui_music` -- Music Generation (not shipped)
### `comfyui_music` -- Music Generation (shipped, custom-workflow-only)
We explored adding a `comfyui_music` tool using the ACE-Step 3.5B model.
The model runs well in ComfyUI, but the ComfyUI node interface for
ACE-Step is not standardized -- there are multiple custom node packs with
different class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`,
etc.). Shipping a workflow that only works with one specific custom node
pack would break for most users.
`tools/audio/comfyui_music.py`. `capability="music_generation"`, `provider="comfyui"`.
Ships with **no bundled workflow** -- the ACE-Step node-pack fragmentation
described below is real and unsolved, so instead of picking one pack and
breaking for everyone else, the tool always requires a caller-supplied
`workflow_json`/`workflow_path` + `output_node`, exactly like the image/video
tools' *optional* override path, just mandatory here. `prompt` is accepted
for provenance/logging only and is never injected into the workflow --
tags/lyrics must already be baked into the graph before calling, same
convention as image/video custom workflows.
**Future path:** ACE-Step support should be revisited once OpenMontage decides
the music-generation routing shape and a portable ComfyUI audio workflow
contract. Current image/video workflow overrides are intentionally scoped to
image and video artifacts, not arbitrary audio workflows.
Originally not shipped because: the ComfyUI node interface for ACE-Step is
not standardized -- there are multiple custom node packs with different
class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.).
Shipping a workflow that only works with one specific custom node pack would
break for most users. The custom-workflow-only design sidesteps this
entirely: whichever node pack is installed, the caller exports it themselves.
**Selector integration:** no dedicated `music_selector` exists in OpenMontage
(unlike `tts_selector`/`image_selector`/`video_selector`) -- music tools are
already routed directly via `registry.get_by_capability("music_generation")`,
and `comfyui_music` participates in that the same way `suno_music`/`music_gen`
do. `fallback_tools = ["suno_music", "music_gen"]`.
**Audio artifact schema:** `ToolResult.data` follows the same shape as the
image/video tools (`provider`, `model`, `output`, `format`, `workflow_provenance`),
plus `duration_seconds` -- a best-effort `ffprobe` probe of the downloaded
file (`None` if `ffprobe` isn't on PATH), since a custom workflow gives no
other reliable way to know actual output duration ahead of time.
**Workflow/output-node contract:** identical to image/video -- `output_node`
must be the ID of the node that writes the final artifact (typically ComfyUI's
native `SaveAudio` node). `ComfyUIClient.generate()`'s artifact extraction now
also checks the `"audio"` output key (previously only `"images"`/`"gifs"`),
which is what `SaveAudio` writes to in ComfyUI's `/history` response --
this is the one part of the contract that *is* standardized regardless of
which ACE-Step loader pack sits upstream of it.
---
@@ -373,15 +398,17 @@ COMFYUI_POLL_TIMEOUT=600 # max wait for image gen
COMFYUI_VIDEO_TIMEOUT=900 # max wait for video gen
```
**Multi-server (optional):** point `comfyui_image` and `comfyui_video` at
separate ComfyUI instances -- e.g. one GPU running FLUX 2, another running
WAN 2.2 -- by setting a per-capability override. Each takes priority over
`COMFYUI_SERVER_URL` for its own tool only; leave both unset and everything
still talks to the single shared server.
**Multi-server (optional):** point `comfyui_image`, `comfyui_video`, and
`comfyui_music` at separate ComfyUI instances -- e.g. one GPU running FLUX 2,
another running WAN 2.2, another running ACE-Step -- by setting a
per-capability override. Each takes priority over `COMFYUI_SERVER_URL` for
its own tool only; leave all three unset and everything talks to the single
shared server.
```bash
COMFYUI_IMAGE_SERVER_URL=http://gpu-a:8188
COMFYUI_VIDEO_SERVER_URL=http://gpu-b:8188
COMFYUI_MUSIC_SERVER_URL=http://gpu-c:8188
```
**For Docker Compose setups** (ComfyUI in a container):
@@ -496,20 +523,23 @@ pipeline definition, or any schema.
`poll()` REST loop when it isn't installed or the connection fails —
`resume_prompt_id` recovery behaves identically either way.
3. ~~**Multi-server:**~~ **Resolved.** `ComfyUIClient(capability="image"|"video")`
3. ~~**Multi-server:**~~ **Resolved.** `ComfyUIClient(capability="image"|"video"|"music")`
resolves its server URL from a per-capability env var first
(`COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL`), then the shared
`COMFYUI_SERVER_URL`, then the `http://localhost:8188` default. `comfyui_image`
and `comfyui_video` pass their capability at construction, so image and video
generation can point at different ComfyUI instances (different GPUs, different
model sets) with zero code changes -- single-server setups need no extra
configuration since both env vars are optional. `client.capability`/
(`COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL` / `COMFYUI_MUSIC_SERVER_URL`),
then the shared `COMFYUI_SERVER_URL`, then the `http://localhost:8188` default.
All three tools pass their capability at construction, so image, video, and
music generation can each point at different ComfyUI instances (different GPUs,
different model sets) with zero code changes -- single-server setups need no extra
configuration since all three env vars are optional. `client.capability`/
`client.is_default_url`/`client.unavailable_reason()` all account for the
override, and `COMFYUI_SETUP_OFFER.per_capability_env_var_overrides` documents
it for the setup-offer surfacing in `provider_menu()`.
4. **Music generation:** ACE-Step works in ComfyUI but OpenMontage needs a
dedicated music-generation routing contract before adding `comfyui_music`.
The follow-up should decide selector integration, audio artifact schemas, and
a portable workflow/output-node contract rather than treating music as a
hidden image/video workflow override.
4. ~~**Music generation:**~~ **Resolved -- shipped as custom-workflow-only.**
`comfyui_music` is a real tool now (not a hidden image/video override), routed
through the existing `registry.get_by_capability("music_generation")` path
like `suno_music`/`music_gen`. It has no bundled workflow -- the node-pack
fragmentation that originally blocked this is real, so the tool always
requires caller-supplied `workflow_json`/`workflow_path` + `output_node`
rather than betting on one pack. See the `comfyui_music` section above for
the artifact schema and workflow/output-node contract.

156
tests/contracts/test_comfyui_tools.py
View File

@@ -17,13 +17,14 @@ from tools.base_tool import (
ToolStatus,
ToolTier,
)
from tools.audio.comfyui_music import ComfyUIMusic
from tools.graphics.comfyui_image import ComfyUIImage
from tools.graphics.image_selector import ImageSelector
from tools.tool_registry import ToolRegistry
from tools.video.video_selector import VideoSelector
from tools.video.comfyui_video import ComfyUIVideo
TOOLS = [ComfyUIImage, ComfyUIVideo]
TOOLS = [ComfyUIImage, ComfyUIVideo, ComfyUIMusic]
WORKFLOW_DIR = Path(__file__).resolve().parent.parent.parent / "tools" / "_comfyui" / "workflows"
PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
@@ -323,6 +324,23 @@ class TestClientHelpers:
)
assert paths == [tmp_path / "out.png"]
def test_generate_reads_audio_key_from_savaudio_node(self, monkeypatch, tmp_path):
"""The native SaveAudio node writes outputs under "audio", not
"images"/"gifs" -- comfyui_music depends on this being handled."""
from tools._comfyui.client import ComfyUIClient
client = ComfyUIClient("http://comfy.test")
monkeypatch.setattr(client, "submit", lambda workflow: "p1")
monkeypatch.setattr(client, "poll", lambda prompt_id, **kwargs: {
"outputs": {"9": {"audio": [{
"filename": "track.flac", "subfolder": "", "type": "output",
}]}}
})
monkeypatch.setattr(client, "download", lambda filename, subfolder, dest, folder_type="output": Path(dest))
paths = client.generate({"9": {"inputs": {}}}, "9", tmp_path / "out.flac")
assert paths == [tmp_path / "out.flac"]
def test_is_default_url_when_env_not_set(self, monkeypatch):
from tools._comfyui.client import ComfyUIClient
monkeypatch.delenv("COMFYUI_SERVER_URL", raising=False)
@@ -838,6 +856,142 @@ class TestCustomWorkflowContract:
assert any(item["role"] == "vae" for item in provenance["model_stack"])
class TestComfyUIMusic:
def test_capability_and_provider(self):
tool = ComfyUIMusic()
assert tool.capability == "music_generation"
assert tool.provider == "comfyui"
def test_requires_workflow_json_or_path(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
result = tool.execute({"prompt": "ambient pad", "output_node": "9"})
assert result.success is False
assert "workflow_json" in result.error or "workflow_path" in result.error
def test_requires_output_node(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
result = tool.execute({
"prompt": "ambient pad",
"workflow_json": json.dumps({"9": {"inputs": {}}}),
})
assert result.success is False
assert "output_node" in result.error
def test_unavailable_server_reports_unavailable_reason(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: False
tool._client.unavailable_reason = lambda: "no server here"
result = tool.execute({
"prompt": "ambient pad",
"workflow_json": json.dumps({"9": {"inputs": {}}}),
"output_node": "9",
})
assert result.success is False
assert result.error == "no server here"
def test_successful_generation_returns_provenance_and_duration(self, tmp_path, monkeypatch):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
dest_file = tmp_path / "music.mp3"
def fake_generate(workflow, output_node, dest, **kwargs):
Path(dest).write_bytes(b"fake-audio-bytes")
return [Path(dest)]
tool._client.generate = fake_generate
monkeypatch.setattr("shutil.which", lambda name: None) # no ffprobe in test env
result = tool.execute({
"prompt": "upbeat synthwave",
"workflow_json": json.dumps({"9": {"inputs": {}}}),
"output_node": "9",
"output_path": str(dest_file),
"workflow_name": "my-ace-step-graph",
"workflow_model": "ace-step-v1-3.5b",
})
assert result.success is True
assert result.data["provider"] == "comfyui"
assert result.data["model"] == "ace-step-v1-3.5b"
assert result.data["output"] == str(dest_file)
assert result.data["format"] == "mp3"
assert result.data["duration_seconds"] is None # ffprobe unavailable
provenance = result.data["workflow_provenance"]
assert provenance["source"] == "user_supplied"
assert provenance["output_node"] == "9"
assert provenance["workflow_hash_sha256"]
def test_timeout_surfaces_resumable_prompt_id(self, tmp_path):
from tools._comfyui.client import ComfyUIError
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
def fake_generate(workflow, output_node, dest, **kwargs):
raise ComfyUIError("timed out", prompt_id="music-prompt-id")
tool._client.generate = fake_generate
result = tool.execute({
"prompt": "ambient pad",
"workflow_json": json.dumps({"9": {"inputs": {}}}),
"output_node": "9",
"output_path": str(tmp_path / "music.mp3"),
})
assert result.success is False
assert result.data["prompt_id"] == "music-prompt-id"
assert "resume_prompt_id" in result.error
def test_passes_timeout_and_resume_prompt_id_through(self, tmp_path):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
seen = {}
def fake_generate(workflow, output_node, dest, **kwargs):
seen.update(kwargs)
return [Path(dest)]
tool._client.generate = fake_generate
tool.execute({
"prompt": "ambient pad",
"workflow_json": json.dumps({"9": {"inputs": {}}}),
"output_node": "9",
"output_path": str(tmp_path / "music.mp3"),
"timeout_seconds": 3600,
"resume_prompt_id": "already-running-id",
})
assert seen["timeout"] == 3600
assert seen["resume_prompt_id"] == "already-running-id"
def test_registry_discovers_comfyui_music_under_music_generation(self):
registry = ToolRegistry()
tool = ComfyUIMusic()
registry.register(tool)
registry._discovered_packages.add("tools")
by_capability = registry.get_by_capability("music_generation")
assert any(t.name == "comfyui_music" for t in by_capability)
def test_uses_music_capability_env_var_for_multi_server(self, monkeypatch):
monkeypatch.delenv("COMFYUI_SERVER_URL", raising=False)
monkeypatch.setenv("COMFYUI_MUSIC_SERVER_URL", "http://music-gpu:8188")
tool = ComfyUIMusic()
assert tool._client.server_url == "http://music-gpu:8188"
class TestComfyUISetupOffer:
def test_provider_menu_summary_includes_structured_setup_offer(self):

9
tools/_comfyui/client.py
View File

@@ -404,8 +404,13 @@ class ComfyUIClient:
outputs = entry.get("outputs", {})
node_output = outputs.get(output_node, {})
# ComfyUI stores images and videos under the "images" key
items = node_output.get("images", []) or node_output.get("gifs", [])
# ComfyUI stores images/video frames under "images", legacy GIFs
# under "gifs", and the native SaveAudio node's output under "audio".
items = (
node_output.get("images", [])
or node_output.get("gifs", [])
or node_output.get("audio", [])
)
if not items:
raise ComfyUIError(
f"No output artifacts on node {output_node}. "

1
tools/_comfyui/metadata.py
View File

@@ -24,6 +24,7 @@ COMFYUI_SETUP_OFFER: dict[str, Any] = {
"per_capability_env_var_overrides": {
"comfyui_image": "COMFYUI_IMAGE_SERVER_URL",
"comfyui_video": "COMFYUI_VIDEO_SERVER_URL",
"comfyui_music": "COMFYUI_MUSIC_SERVER_URL",
},
}

300
tools/audio/comfyui_music.py Normal file
View File

@@ -0,0 +1,300 @@
"""ComfyUI music generation via a local or remote ComfyUI server.
No bundled workflow: ACE-Step's ComfyUI node interface is not standardized
across custom node packs (``AceStepModelLoader`` vs native
``TextEncodeAceStepAudio``, etc.), so a hardcoded template would break for
most installs. This tool always runs a caller-supplied ``workflow_json`` or
``workflow_path`` -- the same override contract ``comfyui_image``/
``comfyui_video`` offer as an alternative to their bundled workflow, just
mandatory here instead of optional. See the ``comfyui`` skill for how to
convert a community ACE-Step workflow into a call.
"""
from __future__ import annotations
import json
import shutil
import subprocess
import time
from pathlib import Path
from typing import Any
from tools.base_tool import (
BaseTool,
Determinism,
ExecutionMode,
ResourceProfile,
RetryPolicy,
ToolResult,
ToolRuntime,
ToolStability,
ToolStatus,
ToolTier,
)
from tools._comfyui.client import ComfyUIClient, ComfyUIError
from tools._comfyui.metadata import COMFYUI_SETUP_OFFER, workflow_hash
class ComfyUIMusic(BaseTool):
name = "comfyui_music"
version = "0.1.0"
tier = ToolTier.GENERATE
capability = "music_generation"
provider = "comfyui"
stability = ToolStability.EXPERIMENTAL
execution_mode = ExecutionMode.SYNC
determinism = Determinism.SEEDED
runtime = ToolRuntime.LOCAL_GPU
dependencies = [] # checked at runtime via server health
setup_offer = COMFYUI_SETUP_OFFER
install_instructions = (
"Start a ComfyUI server with ACE-Step installed (any node pack) and "
"set COMFYUI_SERVER_URL (default http://localhost:8188).\n"
"There is no bundled workflow for this tool -- export your ACE-Step "
"graph in API format and pass it as workflow_json/workflow_path.\n"
"Running a separate ComfyUI instance for music? Set "
"COMFYUI_MUSIC_SERVER_URL instead -- it takes priority over "
"COMFYUI_SERVER_URL for this tool only."
)
agent_skills = ["comfyui"]
capabilities = ["generate_background_music", "generate_song", "generate_instrumental"]
supports = {
"seed": True,
"custom_workflow": True,
"custom_output_node": True,
"offline": True,
}
best_for = [
"local GPU music generation without API costs, using whatever ACE-Step node pack is installed",
"full control over sampling via custom ComfyUI workflows",
]
not_good_for = [
"setups without a running ComfyUI server",
"quick generation without first exporting/adapting an ACE-Step workflow",
"CPU-only machines",
]
fallback_tools = ["suno_music", "music_gen"]
input_schema = {
"type": "object",
"required": ["prompt", "output_node"],
"properties": {
"prompt": {
"type": "string",
"description": (
"Description of the desired music, for provenance/logging only. "
"Not injected into the workflow -- bake the actual tags/lyrics "
"into workflow_json/workflow_path before calling."
),
},
"seed": {"type": "integer", "description": "Random if omitted"},
"output_path": {"type": "string", "description": "Where to save the audio"},
"workflow_json": {
"type": "string",
"description": "Full ComfyUI ACE-Step workflow JSON (API format). Required if workflow_path is omitted.",
},
"workflow_path": {
"type": "string",
"description": "Path to a ComfyUI ACE-Step workflow JSON file. Required if workflow_json is omitted.",
},
"output_node": {
"type": "string",
"description": "ComfyUI output node ID (e.g. the SaveAudio node) to download the artifact from.",
},
"workflow_name": {
"type": "string",
"description": "Optional human-readable provenance label for the workflow.",
},
"workflow_model": {
"type": "string",
"description": "Optional model/provenance label (e.g. 'ace-step-v1-3.5b').",
},
"workflow_model_stack": {
"type": "array",
"description": (
"Optional provenance metadata for workflow dependencies. "
"Items should include name, role, and node-pack origin when known."
),
"items": {"type": "object"},
},
"timeout_seconds": {
"type": "integer",
"description": "How long to wait for the ComfyUI job before giving up. Default 1800s (30min).",
},
"resume_prompt_id": {
"type": "string",
"description": "A prompt_id from a previous timed-out call. Skips resubmission and resumes waiting/downloading.",
},
},
}
resource_profile = ResourceProfile(
cpu_cores=2, ram_mb=8000, vram_mb=8000, disk_mb=500, network_required=False,
)
retry_policy = RetryPolicy(max_retries=1, retryable_errors=["timeout"])
idempotency_key_fields = ["prompt", "seed", "workflow_json", "workflow_path", "output_node"]
side_effects = ["writes audio file to output_path"]
user_visible_verification = ["Listen to generated audio for mood, genre accuracy, and quality"]
def __init__(self) -> None:
self._client = ComfyUIClient(capability="music")
self._last_progress_log = 0.0
def get_status(self) -> ToolStatus:
if not self._client.is_available():
return ToolStatus.UNAVAILABLE
return ToolStatus.AVAILABLE
def estimate_cost(self, inputs: dict[str, Any]) -> float:
return 0.0
def estimate_runtime(self, inputs: dict[str, Any]) -> float:
# Actual runtime depends entirely on the caller's custom workflow
# (steps, duration, sampler); this is a conservative flat estimate.
return 180.0
def get_info(self) -> dict[str, Any]:
info = super().get_info()
info["setup_offer"] = self.setup_offer
info["bundled_workflow"] = None
info["custom_workflow_required"] = True
return info
def _log_progress(self, data: dict) -> None:
"""Throttled progress line (see comfyui_video for rationale)."""
now = time.monotonic()
if now - self._last_progress_log < 10:
return
self._last_progress_log = now
value, max_value = data.get("value"), data.get("max")
if value is not None and max_value:
print(f"[comfyui_music] step {value}/{max_value}")
def execute(self, inputs: dict[str, Any]) -> ToolResult:
if not (inputs.get("workflow_json") or inputs.get("workflow_path")):
return ToolResult(
success=False,
error=(
"comfyui_music requires workflow_json or workflow_path -- there "
"is no bundled default. ACE-Step's ComfyUI node interface isn't "
"standardized across custom node packs, so a hardcoded template "
"would break for most installs. Export the ACE-Step workflow "
"you actually have installed (API format) and pass it in."
),
)
if not inputs.get("output_node"):
return ToolResult(
success=False,
error="output_node is required so OpenMontage knows which ComfyUI node to download the audio from.",
)
if not self._client.is_available():
return ToolResult(success=False, error=self._client.unavailable_reason())
start = time.time()
seed = inputs.get("seed") or ComfyUIClient.random_seed()
output_path = Path(inputs.get("output_path", f"comfyui_music_{seed}.mp3"))
output_node = str(inputs["output_node"])
try:
workflow = self._load_custom_workflow(inputs)
provenance = self._workflow_provenance(inputs, output_node, workflow)
paths = self._client.generate(
workflow,
output_node=output_node,
dest=output_path,
timeout=inputs.get("timeout_seconds", 1800),
interval=10,
resume_prompt_id=inputs.get("resume_prompt_id"),
on_progress=self._log_progress,
)
except ComfyUIError as exc:
data = {"prompt_id": exc.prompt_id} if exc.prompt_id else {}
if exc.prompt_id:
error_msg = (
f"{exc}\n\nThis job was NOT cancelled and is very likely still "
f"running server-side. To recover it without resubmitting, call "
f"execute() again with resume_prompt_id={exc.prompt_id!r} "
f"(and a longer timeout_seconds if it needs more time), or poll "
f"GET {{COMFYUI_SERVER_URL}}/history/{exc.prompt_id} directly."
)
else:
error_msg = str(exc)
return ToolResult(success=False, error=error_msg, data=data)
except Exception as exc:
return ToolResult(success=False, error=f"ComfyUI music generation failed: {exc}")
duration = self._probe_duration(paths[0])
model_name = self._model_name(inputs)
return ToolResult(
success=True,
data={
"provider": "comfyui",
"model": model_name,
"prompt": inputs["prompt"],
"duration_seconds": duration,
"output": str(paths[0]),
"format": paths[0].suffix.lstrip("."),
"workflow_provenance": provenance,
},
artifacts=[str(p) for p in paths],
cost_usd=0.0,
duration_seconds=round(time.time() - start, 2),
seed=seed,
model=model_name,
)
@staticmethod
def _load_custom_workflow(inputs: dict[str, Any]) -> dict:
if inputs.get("workflow_json"):
return json.loads(inputs["workflow_json"])
return ComfyUIClient.load_workflow(Path(inputs["workflow_path"]))
@staticmethod
def _model_name(inputs: dict[str, Any]) -> str:
return (
inputs.get("workflow_model")
or inputs.get("model")
or inputs.get("workflow_name")
or "custom-comfyui-workflow"
)
@staticmethod
def _workflow_provenance(
inputs: dict[str, Any], output_node: str, workflow: dict[str, Any]
) -> dict[str, Any]:
stack = inputs.get("workflow_model_stack")
return {
"source": "user_supplied",
"workflow_name": inputs.get("workflow_name"),
"workflow_path": inputs.get("workflow_path"),
"model": inputs.get("workflow_model") or inputs.get("model"),
"workflow_hash_sha256": workflow_hash(workflow),
"model_stack": stack if isinstance(stack, list) else [],
"model_stack_source": "caller_supplied" if stack else "unknown_custom_workflow",
"output_node": output_node,
}
@staticmethod
def _probe_duration(path: Path) -> float | None:
"""Best-effort track duration via ffprobe; None if unavailable."""
if shutil.which("ffprobe") is None:
return None
try:
out = subprocess.run(
[
"ffprobe", "-v", "error",
"-show_entries", "format=duration",
"-of", "default=noprint_wrappers=1:nokey=1",
str(path),
],
capture_output=True, text=True, timeout=15, check=True,
)
value = out.stdout.strip()
return round(float(value), 2) if value else None
except (subprocess.SubprocessError, ValueError):
return None