diff --git a/.agents/skills/comfyui/SKILL.md b/.agents/skills/comfyui/SKILL.md index ea89cc3e..05bec8c2 100644 --- a/.agents/skills/comfyui/SKILL.md +++ b/.agents/skills/comfyui/SKILL.md @@ -58,7 +58,9 @@ Use this skill before calling `comfyui_image`, `comfyui_video`, or `comfyui_musi ## Music (`comfyui_music`) -- Unlike `comfyui_image`/`comfyui_video`, there is **no bundled workflow**. ACE-Step's ComfyUI node interface isn't standardized across custom node packs (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.), so `workflow_json`/`workflow_path` + `output_node` are always required, not optional. -- `prompt` is provenance/logging only -- it is never injected into the workflow. Bake the actual tags/lyrics into the workflow JSON yourself before calling, the same way you would patch a custom image/video workflow. -- `output_node` should be the node that writes the final audio, typically ComfyUI's native `SaveAudio`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers). -- Provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow -- there's no bundled model stack to fall back on here. +- Bundled default is ACE-Step v1 (3.5B) text-to-audio, built from ComfyUI's *native* `TextEncodeAceStepAudio`/`EmptyAceStepLatentAudio` nodes (core, not a third-party pack) -- unlike ACE-Step 1.5 or other custom node packs, v1's interface is standardized enough to bundle safely. +- `prompt` maps to the bundled workflow's `tags` field (style/genre/mood, e.g. `"upbeat electronic pop, female vocals"`), matching the same "prompt = music description" convention `suno_music` uses. `lyrics` is a separate optional field -- leave empty for instrumental, or use `[verse]`/`[chorus]`/`[bridge]` structure tags and `[zh]`/`[ja]`/`[ko]`-style language-code prefixes for non-English lines. +- `duration_seconds`, `steps`, `cfg`, `lyrics_strength`, and `seed` are patchable on the bundled workflow. Missing `ace_step_v1_3.5b.safetensors` surfaces through the same `data.missing_models[]` contract as image/video. +- Need ACE-Step 1.5, a different node pack, or a non-ACE-Step audio model? Fall back to `workflow_json`/`workflow_path` + `output_node`, exactly like a custom image/video workflow -- in that mode `prompt` becomes provenance/logging only again and must already be baked into the graph. +- `output_node` (bundled or custom) should be the node that writes the final audio -- the bundled workflow's is `SaveAudioMP3`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers). +- For custom workflows, provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow. diff --git a/docs/comfyui-adapter-plan.md b/docs/comfyui-adapter-plan.md index c3d3884f..0e29d601 100644 --- a/docs/comfyui-adapter-plan.md +++ b/docs/comfyui-adapter-plan.md @@ -296,24 +296,33 @@ not promote ComfyUI for an operation whose bundled models are missing. --- -### `comfyui_music` -- Music Generation (shipped, custom-workflow-only) +### `comfyui_music` -- Music Generation (shipped, with a native-node bundled workflow) `tools/audio/comfyui_music.py`. `capability="music_generation"`, `provider="comfyui"`. -Ships with **no bundled workflow** -- the ACE-Step node-pack fragmentation -described below is real and unsolved, so instead of picking one pack and -breaking for everyone else, the tool always requires a caller-supplied -`workflow_json`/`workflow_path` + `output_node`, exactly like the image/video -tools' *optional* override path, just mandatory here. `prompt` is accepted -for provenance/logging only and is never injected into the workflow -- -tags/lyrics must already be baked into the graph before calling, same -convention as image/video custom workflows. -Originally not shipped because: the ComfyUI node interface for ACE-Step is -not standardized -- there are multiple custom node packs with different -class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.). -Shipping a workflow that only works with one specific custom node pack would -break for most users. The custom-workflow-only design sidesteps this -entirely: whichever node pack is installed, the caller exports it themselves. +**Bundled default:** ACE-Step v1 (3.5B) text-to-audio, via `tools/_comfyui/workflows/ace-step-1-t2a.json`. +The node-pack fragmentation that originally blocked this tool (`AceStepModelLoader` +vs native `TextEncodeAceStepAudio`, etc.) turned out to be moot for ACE-Step v1: +ComfyUI ships `TextEncodeAceStepAudio`/`EmptyAceStepLatentAudio` as **native core +nodes** (`comfy_extras/nodes_ace.py`), not a third-party pack, and Comfy-Org's own +[`workflow_templates`](https://github.com/Comfy-Org/workflow_templates) repo bundles +an official ACE-Step-v1 template built entirely from those native nodes plus +long-stable core nodes (`CheckpointLoaderSimple`, `KSampler`, `ModelSamplingSD3`, +`VAEDecodeAudio`, `SaveAudioMP3`). Every node's `class_type` and input names in +`ace-step-1-t2a.json` were cross-checked against ComfyUI's own source +(`comfy_extras/nodes_ace.py`, `nodes_audio.py`, `nodes_latent.py`, `nodes.py`) -- +not guessed from the UI export -- since the UI-format template Comfy-Org ships +isn't directly usable as the API-format JSON this client submits. + +`prompt` maps to ACE-Step's `tags` field (style/genre/mood description, matching +the "prompt = description of desired music" convention `suno_music` already uses). +`lyrics` is a separate optional field (empty for instrumental). `duration_seconds`, +`steps`, `cfg`, `lyrics_strength`, and `seed` are all patchable; `shift` and the +tonemap `multiplier` stay at the official template's defaults. + +Newer/different setups aren't locked out: `workflow_json`/`workflow_path` + +`output_node` still works exactly like the image/video tools' override path -- +for ACE-Step 1.5, a different node pack, or a non-ACE-Step audio model entirely. **Selector integration:** no dedicated `music_selector` exists in OpenMontage (unlike `tts_selector`/`image_selector`/`video_selector`) -- music tools are @@ -323,17 +332,16 @@ do. `fallback_tools = ["suno_music", "music_gen"]`. **Audio artifact schema:** `ToolResult.data` follows the same shape as the image/video tools (`provider`, `model`, `output`, `format`, `workflow_provenance`), -plus `duration_seconds` -- a best-effort `ffprobe` probe of the downloaded -file (`None` if `ffprobe` isn't on PATH), since a custom workflow gives no -other reliable way to know actual output duration ahead of time. +plus `lyrics` and `duration_seconds` -- the latter a best-effort `ffprobe` probe +of the downloaded file (`None` if `ffprobe` isn't on PATH), since even the bundled +workflow doesn't report actual rendered duration back through `/history`. **Workflow/output-node contract:** identical to image/video -- `output_node` -must be the ID of the node that writes the final artifact (typically ComfyUI's -native `SaveAudio` node). `ComfyUIClient.generate()`'s artifact extraction now -also checks the `"audio"` output key (previously only `"images"`/`"gifs"`), -which is what `SaveAudio` writes to in ComfyUI's `/history` response -- -this is the one part of the contract that *is* standardized regardless of -which ACE-Step loader pack sits upstream of it. +must be the ID of the node that writes the final artifact (the bundled workflow's +is `SaveAudioMP3`, ComfyUI's native audio saver). `ComfyUIClient.generate()`'s +artifact extraction now also checks the `"audio"` output key (previously only +`"images"`/`"gifs"`), which is what `SaveAudioMP3`/`SaveAudio` write to in +ComfyUI's `/history` response. --- @@ -535,11 +543,12 @@ pipeline definition, or any schema. override, and `COMFYUI_SETUP_OFFER.per_capability_env_var_overrides` documents it for the setup-offer surfacing in `provider_menu()`. -4. ~~**Music generation:**~~ **Resolved -- shipped as custom-workflow-only.** +4. ~~**Music generation:**~~ **Resolved -- shipped with a bundled ACE-Step v1 workflow.** `comfyui_music` is a real tool now (not a hidden image/video override), routed through the existing `registry.get_by_capability("music_generation")` path - like `suno_music`/`music_gen`. It has no bundled workflow -- the node-pack - fragmentation that originally blocked this is real, so the tool always - requires caller-supplied `workflow_json`/`workflow_path` + `output_node` - rather than betting on one pack. See the `comfyui_music` section above for - the artifact schema and workflow/output-node contract. + like `suno_music`/`music_gen`. The node-pack fragmentation that originally + blocked this turned out not to apply to ACE-Step v1: its ComfyUI nodes are + native core nodes, not a third-party pack, so `ace-step-1-t2a.json` ships as + the default, verified node-by-node against ComfyUI's own source. Custom + `workflow_json`/`workflow_path` + `output_node` remains available for other + versions/packs. See the `comfyui_music` section above for the full contract. diff --git a/tests/contracts/test_comfyui_tools.py b/tests/contracts/test_comfyui_tools.py index 27e32f7c..ac533f30 100644 --- a/tests/contracts/test_comfyui_tools.py +++ b/tests/contracts/test_comfyui_tools.py @@ -137,6 +137,7 @@ EXPECTED_WORKFLOWS = [ "flux2-txt2img.json", "wan22-i2v-4step.json", "wan22-t2v-4step.json", + "ace-step-1-t2a.json", ] @@ -651,6 +652,11 @@ class TestModelRequirements: assert len(_REQUIRED_MODELS_T2V) > 0 assert any("t2v" in m.lower() for m in _REQUIRED_MODELS_T2V) + def test_music_tool_has_required_models(self): + from tools.audio.comfyui_music import _REQUIRED_MODELS + assert len(_REQUIRED_MODELS) > 0 + assert any("ace_step" in m.lower() for m in _REQUIRED_MODELS) + # ------------------------------------------------------------------ # Custom workflow contract and provenance @@ -863,16 +869,23 @@ class TestComfyUIMusic: assert tool.capability == "music_generation" assert tool.provider == "comfyui" - def test_requires_workflow_json_or_path(self): + def test_bundled_path_requires_no_workflow_json_or_output_node(self, tmp_path): + """Without workflow_json/workflow_path it should attempt the bundled + ACE-Step workflow, not demand a custom one.""" tool = ComfyUIMusic() tool._client.is_available = lambda: True + tool._client.check_models = lambda required: (list(required), []) + tool._client.generate = lambda workflow, output_node, dest, **kwargs: [Path(dest)] - result = tool.execute({"prompt": "ambient pad", "output_node": "9"}) + result = tool.execute({ + "prompt": "ambient pad", + "output_path": str(tmp_path / "music.mp3"), + }) - assert result.success is False - assert "workflow_json" in result.error or "workflow_path" in result.error + assert result.success is True + assert result.data["workflow_provenance"]["source"] == "bundled" - def test_requires_output_node(self): + def test_custom_workflow_without_output_node_errors(self): tool = ComfyUIMusic() tool._client.is_available = lambda: True @@ -884,6 +897,52 @@ class TestComfyUIMusic: assert result.success is False assert "output_node" in result.error + def test_bundled_missing_models_returns_structured_payload(self): + tool = ComfyUIMusic() + tool._client.is_available = lambda: True + tool._client.check_models = lambda required: ([], list(required)) + + result = tool.execute({"prompt": "ambient pad"}) + + assert result.success is False + assert result.data["missing_models"][0]["name"] == "ace_step_v1_3.5b.safetensors" + assert result.data["missing_models"][0]["download_url"] + + def test_bundled_generation_patches_tags_lyrics_and_seed(self, tmp_path): + tool = ComfyUIMusic() + tool._client.is_available = lambda: True + tool._client.check_models = lambda required: (list(required), []) + seen = {} + + def fake_generate(workflow, output_node, dest, **kwargs): + seen["workflow"] = workflow + seen["output_node"] = output_node + return [Path(dest)] + + tool._client.generate = fake_generate + + result = tool.execute({ + "prompt": "lofi hip hop, chill, rain sounds", + "lyrics": "[verse]\nquiet streets", + "duration_seconds": 45, + "seed": 777, + "output_path": str(tmp_path / "music.mp3"), + }) + + assert result.success is True + assert seen["output_node"] == "10" + assert seen["workflow"]["2"]["inputs"]["tags"] == "lofi hip hop, chill, rain sounds" + assert seen["workflow"]["2"]["inputs"]["lyrics"] == "[verse]\nquiet streets" + assert seen["workflow"]["4"]["inputs"]["seconds"] == 45 + assert seen["workflow"]["8"]["inputs"]["seed"] == 777 + assert result.data["model"] == "ace-step-v1-3.5b" + + def test_get_status_degraded_when_model_missing(self): + tool = ComfyUIMusic() + tool._client.is_available = lambda: True + tool._client.check_models = lambda required: ([], list(required)) + assert tool.get_status() == ToolStatus.DEGRADED + def test_unavailable_server_reports_unavailable_reason(self): tool = ComfyUIMusic() tool._client.is_available = lambda: False diff --git a/tools/_comfyui/metadata.py b/tools/_comfyui/metadata.py index 3c0ee13b..084fbeb3 100644 --- a/tools/_comfyui/metadata.py +++ b/tools/_comfyui/metadata.py @@ -184,6 +184,17 @@ BUNDLED_MODEL_STACKS: dict[str, list[dict[str, Any]]] = { ), }, ], + "ace-step-1-t2a": [ + { + "role": "checkpoint", + "name": "ace_step_v1_3.5b.safetensors", + "destination_hint": "ComfyUI/models/checkpoints/", + "download_url": ( + "https://huggingface.co/Comfy-Org/ACE-Step_ComfyUI_repackaged/" + "blob/main/all_in_one/ace_step_v1_3.5b.safetensors" + ), + }, + ], } diff --git a/tools/_comfyui/workflows/ace-step-1-t2a.json b/tools/_comfyui/workflows/ace-step-1-t2a.json new file mode 100644 index 00000000..fdeba405 --- /dev/null +++ b/tools/_comfyui/workflows/ace-step-1-t2a.json @@ -0,0 +1,80 @@ +{ + "1": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "ace_step_v1_3.5b.safetensors" + } + }, + "2": { + "class_type": "TextEncodeAceStepAudio", + "inputs": { + "clip": ["1", 1], + "tags": "", + "lyrics": "", + "lyrics_strength": 0.99 + } + }, + "3": { + "class_type": "ConditioningZeroOut", + "inputs": { + "conditioning": ["2", 0] + } + }, + "4": { + "class_type": "EmptyAceStepLatentAudio", + "inputs": { + "seconds": 120, + "batch_size": 1 + } + }, + "5": { + "class_type": "ModelSamplingSD3", + "inputs": { + "model": ["1", 0], + "shift": 5.0 + } + }, + "6": { + "class_type": "LatentOperationTonemapReinhard", + "inputs": { + "multiplier": 1.0 + } + }, + "7": { + "class_type": "LatentApplyOperationCFG", + "inputs": { + "model": ["5", 0], + "operation": ["6", 0] + } + }, + "8": { + "class_type": "KSampler", + "inputs": { + "model": ["7", 0], + "positive": ["2", 0], + "negative": ["3", 0], + "latent_image": ["4", 0], + "seed": 0, + "steps": 50, + "cfg": 5.0, + "sampler_name": "euler", + "scheduler": "simple", + "denoise": 1.0 + } + }, + "9": { + "class_type": "VAEDecodeAudio", + "inputs": { + "samples": ["8", 0], + "vae": ["1", 2] + } + }, + "10": { + "class_type": "SaveAudioMP3", + "inputs": { + "audio": ["9", 0], + "filename_prefix": "openmontage", + "quality": "V0" + } + } +} diff --git a/tools/audio/comfyui_music.py b/tools/audio/comfyui_music.py index 1d006f77..9751ad46 100644 --- a/tools/audio/comfyui_music.py +++ b/tools/audio/comfyui_music.py @@ -1,13 +1,11 @@ """ComfyUI music generation via a local or remote ComfyUI server. -No bundled workflow: ACE-Step's ComfyUI node interface is not standardized -across custom node packs (``AceStepModelLoader`` vs native -``TextEncodeAceStepAudio``, etc.), so a hardcoded template would break for -most installs. This tool always runs a caller-supplied ``workflow_json`` or -``workflow_path`` -- the same override contract ``comfyui_image``/ -``comfyui_video`` offer as an alternative to their bundled workflow, just -mandatory here instead of optional. See the ``comfyui`` skill for how to -convert a community ACE-Step workflow into a call. +Default workflow: ACE-Step v1 (3.5B) text-to-audio using ComfyUI's native +``TextEncodeAceStepAudio``/``EmptyAceStepLatentAudio`` nodes (built into +ComfyUI core, not a third-party pack). Custom workflows are still accepted +via ``workflow_json``/``workflow_path`` for other ACE-Step node packs, other +versions (e.g. ACE-Step 1.5), or entirely different audio models -- the same +override contract ``comfyui_image``/``comfyui_video`` offer. """ from __future__ import annotations @@ -32,12 +30,23 @@ from tools.base_tool import ( ToolTier, ) from tools._comfyui.client import ComfyUIClient, ComfyUIError -from tools._comfyui.metadata import COMFYUI_SETUP_OFFER, workflow_hash +from tools._comfyui.metadata import ( + BUNDLED_MODEL_STACKS, + COMFYUI_SETUP_OFFER, + missing_models_payload, + model_stack, + workflow_hash, +) + +_WORKFLOWS = Path(__file__).resolve().parent.parent / "_comfyui" / "workflows" + +# Model required by the bundled ACE-Step v1 workflow +_REQUIRED_MODELS = ["ace_step_v1_3.5b.safetensors"] class ComfyUIMusic(BaseTool): name = "comfyui_music" - version = "0.1.0" + version = "0.2.0" tier = ToolTier.GENERATE capability = "music_generation" provider = "comfyui" @@ -49,10 +58,10 @@ class ComfyUIMusic(BaseTool): dependencies = [] # checked at runtime via server health setup_offer = COMFYUI_SETUP_OFFER install_instructions = ( - "Start a ComfyUI server with ACE-Step installed (any node pack) and " - "set COMFYUI_SERVER_URL (default http://localhost:8188).\n" - "There is no bundled workflow for this tool -- export your ACE-Step " - "graph in API format and pass it as workflow_json/workflow_path.\n" + "Start a ComfyUI server and set COMFYUI_SERVER_URL " + "(default http://localhost:8188).\n" + "Requires ace_step_v1_3.5b.safetensors in ComfyUI's checkpoints " + "directory for the bundled workflow.\n" "Running a separate ComfyUI instance for music? Set " "COMFYUI_MUSIC_SERVER_URL instead -- it takes priority over " "COMFYUI_SERVER_URL for this tool only." @@ -62,59 +71,73 @@ class ComfyUIMusic(BaseTool): capabilities = ["generate_background_music", "generate_song", "generate_instrumental"] supports = { "seed": True, + "lyrics": True, "custom_workflow": True, "custom_output_node": True, "offline": True, } best_for = [ - "local GPU music generation without API costs, using whatever ACE-Step node pack is installed", - "full control over sampling via custom ComfyUI workflows", + "local GPU music generation without API costs", + "instrumentals and songs with lyrics via the bundled ACE-Step v1 workflow", + "full control over sampling or other ACE-Step versions/node packs via custom ComfyUI workflows", ] not_good_for = [ "setups without a running ComfyUI server", - "quick generation without first exporting/adapting an ACE-Step workflow", "CPU-only machines", ] fallback_tools = ["suno_music", "music_gen"] input_schema = { "type": "object", - "required": ["prompt", "output_node"], + "required": ["prompt"], "properties": { "prompt": { "type": "string", "description": ( - "Description of the desired music, for provenance/logging only. " - "Not injected into the workflow -- bake the actual tags/lyrics " - "into workflow_json/workflow_path before calling." + "Style/mood/genre description (ACE-Step 'tags'), e.g. " + "'upbeat electronic pop, female vocals, driving bassline'. " + "Comma-separated tags work best. Not injected for custom workflows." ), }, + "lyrics": { + "type": "string", + "default": "", + "description": ( + "Optional lyrics. Leave empty for instrumental. Supports structure " + "tags like [verse]/[chorus]/[bridge] and language-code prefixes " + "(e.g. [zh], [ja]) for non-English lines." + ), + }, + "duration_seconds": {"type": "number", "default": 120.0}, + "steps": {"type": "integer", "default": 50}, + "cfg": {"type": "number", "default": 5.0}, + "lyrics_strength": {"type": "number", "default": 0.99}, "seed": {"type": "integer", "description": "Random if omitted"}, "output_path": {"type": "string", "description": "Where to save the audio"}, "workflow_json": { "type": "string", - "description": "Full ComfyUI ACE-Step workflow JSON (API format). Required if workflow_path is omitted.", + "description": "Optional full ComfyUI workflow JSON. Requires output_node.", }, "workflow_path": { "type": "string", - "description": "Path to a ComfyUI ACE-Step workflow JSON file. Required if workflow_json is omitted.", + "description": "Optional path to a ComfyUI workflow JSON file. Requires output_node.", }, "output_node": { "type": "string", - "description": "ComfyUI output node ID (e.g. the SaveAudio node) to download the artifact from.", + "description": "ComfyUI output node ID for custom workflow_json/workflow_path.", }, "workflow_name": { "type": "string", - "description": "Optional human-readable provenance label for the workflow.", + "description": "Optional human-readable provenance label for a custom workflow.", }, "workflow_model": { "type": "string", - "description": "Optional model/provenance label (e.g. 'ace-step-v1-3.5b').", + "description": "Optional model/provenance label for a custom workflow.", }, "workflow_model_stack": { "type": "array", "description": ( - "Optional provenance metadata for workflow dependencies. " + "Optional provenance metadata for custom workflow dependencies. " "Items should include name, role, and node-pack origin when known." ), "items": {"type": "object"}, @@ -134,7 +157,7 @@ class ComfyUIMusic(BaseTool): cpu_cores=2, ram_mb=8000, vram_mb=8000, disk_mb=500, network_required=False, ) retry_policy = RetryPolicy(max_retries=1, retryable_errors=["timeout"]) - idempotency_key_fields = ["prompt", "seed", "workflow_json", "workflow_path", "output_node"] + idempotency_key_fields = ["prompt", "lyrics", "duration_seconds", "seed"] side_effects = ["writes audio file to output_path"] user_visible_verification = ["Listen to generated audio for mood, genre accuracy, and quality"] @@ -145,21 +168,21 @@ class ComfyUIMusic(BaseTool): def get_status(self) -> ToolStatus: if not self._client.is_available(): return ToolStatus.UNAVAILABLE + _, missing = self._client.check_models(_REQUIRED_MODELS) + if missing: + return ToolStatus.DEGRADED return ToolStatus.AVAILABLE def estimate_cost(self, inputs: dict[str, Any]) -> float: return 0.0 def estimate_runtime(self, inputs: dict[str, Any]) -> float: - # Actual runtime depends entirely on the caller's custom workflow - # (steps, duration, sampler); this is a conservative flat estimate. - return 180.0 + return float(inputs.get("steps", 50)) * 2.0 def get_info(self) -> dict[str, Any]: info = super().get_info() info["setup_offer"] = self.setup_offer - info["bundled_workflow"] = None - info["custom_workflow_required"] = True + info["bundled_model_stack"] = BUNDLED_MODEL_STACKS["ace-step-1-t2a"] return info def _log_progress(self, data: dict) -> None: @@ -173,35 +196,63 @@ class ComfyUIMusic(BaseTool): print(f"[comfyui_music] step {value}/{max_value}") def execute(self, inputs: dict[str, Any]) -> ToolResult: - if not (inputs.get("workflow_json") or inputs.get("workflow_path")): + custom_workflow = bool(inputs.get("workflow_json") or inputs.get("workflow_path")) + if custom_workflow and not inputs.get("output_node"): return ToolResult( success=False, error=( - "comfyui_music requires workflow_json or workflow_path -- there " - "is no bundled default. ACE-Step's ComfyUI node interface isn't " - "standardized across custom node packs, so a hardcoded template " - "would break for most installs. Export the ACE-Step workflow " - "you actually have installed (API format) and pass it in." + "Custom ComfyUI workflows require output_node so OpenMontage " + "knows which ComfyUI node to download artifacts from." ), ) - if not inputs.get("output_node"): - return ToolResult( - success=False, - error="output_node is required so OpenMontage knows which ComfyUI node to download the audio from.", - ) - if not self._client.is_available(): return ToolResult(success=False, error=self._client.unavailable_reason()) + if not custom_workflow: + _, missing = self._client.check_models(_REQUIRED_MODELS) + if missing: + return ToolResult( + success=False, + data=missing_models_payload( + missing, + workflow_key="ace-step-1-t2a", + workflow_name="ace-step-1-t2a.json", + ), + error=( + f"ComfyUI server is running but missing required models: " + f"{', '.join(missing)}.\n" + f"See data.missing_models for destination hints and download URLs." + ), + ) + start = time.time() seed = inputs.get("seed") or ComfyUIClient.random_seed() output_path = Path(inputs.get("output_path", f"comfyui_music_{seed}.mp3")) - output_node = str(inputs["output_node"]) try: - workflow = self._load_custom_workflow(inputs) - provenance = self._workflow_provenance(inputs, output_node, workflow) + if custom_workflow: + workflow = self._load_custom_workflow(inputs) + output_node = str(inputs["output_node"]) + else: + workflow = ComfyUIClient.load_workflow(_WORKFLOWS / "ace-step-1-t2a.json") + workflow = ComfyUIClient.patch_workflow(workflow, { + "2": { + "tags": inputs["prompt"], + "lyrics": inputs.get("lyrics", ""), + "lyrics_strength": inputs.get("lyrics_strength", 0.99), + }, + "4": {"seconds": inputs.get("duration_seconds", 120.0)}, + "8": { + "seed": seed, + "steps": inputs.get("steps", 50), + "cfg": inputs.get("cfg", 5.0), + }, + "10": {"filename_prefix": output_path.stem}, + }) + output_node = "10" + + provenance = self._workflow_provenance(inputs, custom_workflow, output_node, workflow) paths = self._client.generate( workflow, output_node=output_node, @@ -229,13 +280,14 @@ class ComfyUIMusic(BaseTool): return ToolResult(success=False, error=f"ComfyUI music generation failed: {exc}") duration = self._probe_duration(paths[0]) - model_name = self._model_name(inputs) + model_name = self._model_name(inputs, custom_workflow) return ToolResult( success=True, data={ "provider": "comfyui", "model": model_name, "prompt": inputs["prompt"], + "lyrics": inputs.get("lyrics", ""), "duration_seconds": duration, "output": str(paths[0]), "format": paths[0].suffix.lstrip("."), @@ -255,7 +307,9 @@ class ComfyUIMusic(BaseTool): return ComfyUIClient.load_workflow(Path(inputs["workflow_path"])) @staticmethod - def _model_name(inputs: dict[str, Any]) -> str: + def _model_name(inputs: dict[str, Any], custom_workflow: bool) -> str: + if not custom_workflow: + return "ace-step-v1-3.5b" return ( inputs.get("workflow_model") or inputs.get("model") @@ -265,8 +319,19 @@ class ComfyUIMusic(BaseTool): @staticmethod def _workflow_provenance( - inputs: dict[str, Any], output_node: str, workflow: dict[str, Any] + inputs: dict[str, Any], + custom_workflow: bool, + output_node: str, + workflow: dict[str, Any], ) -> dict[str, Any]: + if not custom_workflow: + return { + "source": "bundled", + "workflow": "ace-step-1-t2a.json", + "workflow_hash_sha256": workflow_hash(workflow), + "model_stack": model_stack("ace-step-1-t2a", inputs), + "output_node": output_node, + } stack = inputs.get("workflow_model_stack") return { "source": "user_supplied",