feat: add OpenClaw compatibility layer

- Add openclaw/ directory with workspace files (AGENTS.md, SOUL.md,
  TOOLS.md), setup script, and config template
- Add OpenClaw metadata (emoji, requirements, env vars) to all 12
  SKILL.md frontmatter blocks for cross-platform compatibility
- Add OpenClaw section to README with setup instructions
- Both Claude Code and OpenClaw follow the AgentSkills spec, so
  skills work on either platform with minimal differences
This commit is contained in:
MCKRUZ
2026-02-06 16:41:30 -05:00
parent 469f480f71
commit e979cf66df
18 changed files with 426 additions and 0 deletions
+84
View File
@@ -305,6 +305,13 @@ ComfyUI-Expert/
|-- agent/
| +-- AGENT.md Extended orchestration spec
|
|-- openclaw/ OpenClaw compatibility layer
| |-- AGENTS.md Orchestration rules (CLAUDE.md equivalent)
| |-- SOUL.md Agent persona
| |-- TOOLS.md Available tools & API reference
| |-- setup.ps1 Install skills into OpenClaw workspace
| +-- openclaw.example.json Config template
|
+-- docs/
|-- architecture.md System design decisions
+-- getting-started.md Quick start guide
@@ -396,6 +403,83 @@ Update the PowerShell script paths in `.claude/settings.local.json` to use `pwsh
| Staleness hook not firing | Check `.claude/settings.local.json` is valid JSON |
| Skills leaking to other sessions | They shouldn't -- skills are local files, not globally installed |
## OpenClaw Compatibility
VideoAgent skills also work with [OpenClaw](https://github.com/openclaw/openclaw). Both platforms follow the [AgentSkills specification](https://agentskills.io), so the skill files are cross-compatible. The `openclaw/` directory contains everything needed.
### How it maps
| Claude Code | OpenClaw Equivalent | Notes |
|-------------|--------------------|----|
| `CLAUDE.md` (orchestrator) | `AGENTS.md` + `SOUL.md` + `TOOLS.md` | Split across three workspace files |
| `video-agent.bat` (launcher) | OpenClaw daemon | OpenClaw runs as a persistent service |
| `.claude/settings.local.json` (hooks) | `openclaw.json` (config) | Different config format |
| Auto-loads from project root | Skills in `~/.openclaw/workspace/skills/` | Must be installed to workspace |
| `{skill}/SKILL.md` frontmatter | Same + `metadata.openclaw` block | Already added to all 12 skills |
### Setup
```powershell
# 1. Run the setup script (copies skills + workspace files into OpenClaw)
pwsh -File openclaw/setup.ps1
# Or use symlinks to keep them in sync with the repo
pwsh -File openclaw/setup.ps1 -Symlink
# Or specify a custom OpenClaw workspace path
pwsh -File openclaw/setup.ps1 -OpenClawDir "~/.openclaw/workspace"
```
```powershell
# 2. Add skill config to your ~/.openclaw/openclaw.json
# See openclaw/openclaw.example.json for the full template -- at minimum:
```
```json
{
"skills": {
"entries": {
"comfyui-api": {
"enabled": true,
"env": { "COMFYUI_URL": "http://127.0.0.1:8188" }
},
"comfyui-inventory": {
"enabled": true,
"env": { "COMFYUI_PATH": "C:\\ComfyUI" }
}
}
}
}
```
```powershell
# 3. Restart OpenClaw to pick up the new skills
```
### What the setup script does
1. Copies (or symlinks) all 12 skill folders into `~/.openclaw/workspace/skills/`
2. Copies `AGENTS.md`, `SOUL.md`, `TOOLS.md` into the workspace root
3. Copies `foundation/` and `references/` alongside skills for reference access
### What's different in OpenClaw
- **Skill routing**: Claude Code uses a routing table in `CLAUDE.md`. OpenClaw uses keyword matching on the `description` field in each skill's frontmatter -- the descriptions are already written to support this.
- **Requirements gating**: OpenClaw validates `metadata.openclaw.requires` at load time (checks that binaries/env vars exist). Skills that fail requirements are excluded from the session.
- **No session hook**: The staleness-check hook is Claude Code specific. In OpenClaw, ask the agent to check for stale research manually, or set up a cron job.
- **File references**: OpenClaw skills can use `{baseDir}` to reference files relative to the skill folder. The current skills use relative paths that work in both environments.
### Files
```
openclaw/
|-- AGENTS.md Orchestration rules (CLAUDE.md equivalent)
|-- SOUL.md Agent persona
|-- TOOLS.md Available tools and API reference
|-- setup.ps1 Installation script
+-- openclaw.example.json Config template for ~/.openclaw/openclaw.json
```
## License
MIT
+62
View File
@@ -0,0 +1,62 @@
# VideoAgent Orchestration
You are running the **VideoAgent** skill set for ComfyUI video production. These skills work together as a pipeline. Follow the rules below when handling requests.
## Critical Rule: Always Check Inventory First
Before generating ANY ComfyUI workflow:
1. Use the `comfyui-inventory` skill to verify `state/inventory.json` exists
2. If it doesn't exist, tell the user to run the scan script
3. Validate every model and node in your workflow exists in the inventory
4. If something is missing, say what to download and where to put it
## Skill Routing
When the user makes a request, the most relevant skill should handle it. Here's how requests map:
| User Wants | Skill |
|------------|-------|
| Generate/create character image | `comfyui-workflow-builder` |
| Craft or optimize prompts | `comfyui-prompt-engineer` |
| Create video / animate | `comfyui-video-pipeline` |
| Clone voice / generate speech | `comfyui-voice-pipeline` |
| Train a LoRA | `comfyui-lora-training` |
| Build raw ComfyUI workflow | `comfyui-workflow-builder` |
| Research latest models | `comfyui-research` |
| Something broke / error | `comfyui-troubleshooter` |
| Assemble final video | `video-assembly` |
| Upload / publish | `video-publisher` |
| Manage project / characters | `project-manager` |
| Connect to ComfyUI / check status | `comfyui-api` |
| Check what's installed | `comfyui-inventory` |
## Multi-Step Pipeline Pattern
For complex requests (e.g., "make a talking head video"):
1. **Gather context**: Check inventory + project state
2. **Plan the pipeline**: Identify all steps, tell the user the plan
3. **Execute in order**: Use each skill as needed
4. **Validate outputs**: Check results before proceeding
5. **Update state**: Note what worked in project notes
## Authority Matrix
| Decision | Agent Decides | Ask User |
|----------|:---:|:---:|
| Which workflow pattern to use | X | |
| Model selection (clear best option) | X | |
| Model selection (tradeoffs involved) | | X |
| VRAM optimization flags | X | |
| LoRA training hyperparameters | | X |
| Voice selection / clone source | | X |
| Publishing targets | | X |
| Spending money (API calls, cloud GPU) | | X |
## Error Recovery
When something fails:
1. Use the `comfyui-troubleshooter` skill
2. Match the error pattern
3. If missing model/node: suggest download from the models reference
4. If VRAM issue: suggest optimization flags or model swap
+19
View File
@@ -0,0 +1,19 @@
# VideoAgent Persona
You are **VideoAgent**, a senior AI video production technical director specializing in ComfyUI-based pipelines.
## Communication Style
- Direct and practical -- recommend proven approaches first
- Explain tradeoffs concisely when multiple options exist
- Always mention VRAM requirements upfront
- Flag when something is experimental vs battle-tested
- Use specific model names and exact settings (no vague "try adjusting")
## Principles
1. **Verify before generating**: Always check inventory before building workflows
2. **Hardware-aware**: Optimize for available GPU but note alternatives for other VRAM tiers
3. **Project continuity**: Reference past successful settings from project manifests
4. **Honest about limitations**: If a technique hasn't been tested, say so
5. **Incremental complexity**: Start with simpler pipeline, add complexity only if needed
+50
View File
@@ -0,0 +1,50 @@
# VideoAgent Tools
## ComfyUI REST API
Base URL: configured via `COMFYUI_URL` environment variable (default `http://127.0.0.1:8188`).
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/system_stats` | GET | GPU info, VRAM usage, ComfyUI version |
| `/queue` | GET | Current queue status |
| `/interrupt` | POST | Cancel current generation |
| `/free` | POST | Free VRAM (`{"unload_models": true}`) |
| `/object_info` | GET | All installed node classes |
| `/models/{type}` | GET | List models (checkpoints, loras, vae, controlnet, etc.) |
| `/prompt` | POST | Queue a workflow (`{"prompt": {...}}`) |
| `/history/{prompt_id}` | GET | Execution result for a queued workflow |
| `/view` | GET | Retrieve output image (`?filename=...&type=output`) |
| `/upload/image` | POST | Upload image (multipart) |
## Polling Pattern
```bash
# 1. Queue workflow
curl -s -X POST $COMFYUI_URL/prompt -H "Content-Type: application/json" -d '{"prompt": WORKFLOW_JSON}'
# Returns: {"prompt_id": "abc-123"}
# 2. Poll for completion (every 5s)
curl -s $COMFYUI_URL/history/abc-123
# When complete: {"abc-123": {"outputs": {...}, "status": {"completed": true}}}
# 3. Retrieve output
curl -s "$COMFYUI_URL/view?filename=output.png&subfolder=&type=output" --output result.png
```
## Scripts
All scripts are in the repo's `scripts/` directory. When installed via `setup.ps1`, they're accessible at `{baseDir}/../../scripts/`.
| Script | Purpose |
|--------|---------|
| `scan-inventory.ps1` | Scan ComfyUI models & nodes offline |
| `connect-comfyui.ps1` | Test ComfyUI connection & diagnostics |
## FFmpeg (for video assembly)
Required for video-assembly skill. Must be on PATH.
```bash
ffmpeg -version
```
+64
View File
@@ -0,0 +1,64 @@
{
"skills": {
"entries": {
"comfyui-api": {
"enabled": true,
"env": {
"COMFYUI_URL": "http://127.0.0.1:8188"
}
},
"comfyui-inventory": {
"enabled": true,
"env": {
"COMFYUI_PATH": "C:\\ComfyUI"
}
},
"comfyui-workflow-builder": {
"enabled": true,
"env": {
"COMFYUI_URL": "http://127.0.0.1:8188",
"COMFYUI_PATH": "C:\\ComfyUI"
}
},
"comfyui-prompt-engineer": {
"enabled": true
},
"comfyui-video-pipeline": {
"enabled": true,
"env": {
"COMFYUI_URL": "http://127.0.0.1:8188"
}
},
"comfyui-voice-pipeline": {
"enabled": true,
"env": {
"COMFYUI_URL": "http://127.0.0.1:8188"
}
},
"comfyui-lora-training": {
"enabled": true,
"env": {
"COMFYUI_PATH": "C:\\ComfyUI"
}
},
"comfyui-research": {
"enabled": true
},
"comfyui-troubleshooter": {
"enabled": true
},
"project-manager": {
"enabled": true
},
"video-assembly": {
"enabled": true
},
"video-publisher": {
"enabled": true
}
},
"load": {
"extraDirs": []
}
}
}
+123
View File
@@ -0,0 +1,123 @@
<#
.SYNOPSIS
Sets up VideoAgent skills for OpenClaw.
.DESCRIPTION
Copies (or symlinks) the 12 VideoAgent skills plus workspace files
into OpenClaw's directory structure. Run once after cloning.
.PARAMETER OpenClawDir
Path to your OpenClaw workspace (default: ~/.openclaw/workspace)
.PARAMETER Symlink
Use symlinks instead of copies. Keeps skills in sync with the repo
but requires running as administrator on Windows.
.EXAMPLE
pwsh -File openclaw/setup.ps1
pwsh -File openclaw/setup.ps1 -OpenClawDir "~/.openclaw/workspace"
pwsh -File openclaw/setup.ps1 -Symlink
#>
param(
[string]$OpenClawDir = (Join-Path $HOME ".openclaw" "workspace"),
[switch]$Symlink
)
$ErrorActionPreference = "Stop"
$RepoRoot = Split-Path -Parent (Split-Path -Parent $PSCommandPath)
$SkillsSource = Join-Path $RepoRoot "skills"
$SkillsDest = Join-Path $OpenClawDir "skills"
$WorkspaceFiles = @("AGENTS.md", "SOUL.md", "TOOLS.md")
# Verify source exists
if (-not (Test-Path $SkillsSource)) {
Write-Error "Skills directory not found at: $SkillsSource"
exit 1
}
Write-Host ""
Write-Host " VideoAgent OpenClaw Setup" -ForegroundColor Cyan
Write-Host " =========================" -ForegroundColor Cyan
Write-Host " Repo: $RepoRoot"
Write-Host " Target: $OpenClawDir"
Write-Host " Mode: $(if ($Symlink) { 'Symlink' } else { 'Copy' })"
Write-Host ""
# Create target directories
if (-not (Test-Path $SkillsDest)) {
New-Item -ItemType Directory -Path $SkillsDest -Force | Out-Null
Write-Host " Created: $SkillsDest" -ForegroundColor Green
}
# Copy or symlink each skill
$skills = Get-ChildItem -Path $SkillsSource -Directory
foreach ($skill in $skills) {
$dest = Join-Path $SkillsDest $skill.Name
if (Test-Path $dest) {
if ($Symlink) {
# Remove existing to recreate symlink
Remove-Item $dest -Recurse -Force
}
else {
Remove-Item $dest -Recurse -Force
}
}
if ($Symlink) {
New-Item -ItemType SymbolicLink -Path $dest -Target $skill.FullName | Out-Null
Write-Host " Linked: $($skill.Name) -> $($skill.FullName)" -ForegroundColor Green
}
else {
Copy-Item -Path $skill.FullName -Destination $dest -Recurse
Write-Host " Copied: $($skill.Name)" -ForegroundColor Green
}
}
# Copy workspace files (AGENTS.md, SOUL.md, TOOLS.md)
$openclawSource = Join-Path $RepoRoot "openclaw"
foreach ($file in $WorkspaceFiles) {
$src = Join-Path $openclawSource $file
$dst = Join-Path $OpenClawDir $file
if (Test-Path $src) {
Copy-Item -Path $src -Destination $dst -Force
Write-Host " Copied: $file -> $OpenClawDir" -ForegroundColor Green
}
else {
Write-Host " Missing: $file (skipped)" -ForegroundColor Yellow
}
}
# Copy foundation and references alongside skills for {baseDir} access
$sharedDirs = @("foundation", "references")
foreach ($dir in $sharedDirs) {
$src = Join-Path $RepoRoot $dir
$dst = Join-Path $OpenClawDir $dir
if (Test-Path $src) {
if (Test-Path $dst) {
Remove-Item $dst -Recurse -Force
}
if ($Symlink) {
New-Item -ItemType SymbolicLink -Path $dst -Target $src | Out-Null
Write-Host " Linked: $dir -> $src" -ForegroundColor Green
}
else {
Copy-Item -Path $src -Destination $dst -Recurse
Write-Host " Copied: $dir/" -ForegroundColor Green
}
}
}
Write-Host ""
Write-Host " Setup complete! $($skills.Count) skills installed." -ForegroundColor Cyan
Write-Host ""
Write-Host " Next steps:" -ForegroundColor Yellow
Write-Host " 1. Copy openclaw/openclaw.example.json settings into ~/.openclaw/openclaw.json"
Write-Host " 2. Edit COMFYUI_URL and COMFYUI_PATH to match your setup"
Write-Host " 3. Restart OpenClaw to pick up the new skills"
Write-Host ""
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-api
description: Connect to a running ComfyUI instance, queue workflows, monitor execution, and retrieve results. Supports both online (REST API) and offline (JSON export) modes. Use when executing ComfyUI workflows or checking server status.
user-invocable: true
metadata: {"openclaw":{"emoji":"🔌","os":["darwin","linux","win32"],"requires":{"anyBins":["curl","wget"]},"primaryEnv":"COMFYUI_URL"}}
---
# ComfyUI API Skill
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-inventory
description: Discover and cache all installed ComfyUI models, custom nodes, and system capabilities. Works online (API queries) and offline (directory scanning). Use before generating workflows to verify available resources.
user-invocable: true
metadata: {"openclaw":{"emoji":"📦","os":["darwin","linux","win32"],"requires":{"bins":["pwsh"]},"primaryEnv":"COMFYUI_PATH"}}
---
# ComfyUI Inventory Skill
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-lora-training
description: Prepare datasets and configure LoRA training for character consistency. Covers FLUX (AI-Toolkit, SimpleTuner, FluxGym) and SDXL (Kohya_ss) training with step-by-step guidance. Use when training custom character LoRAs.
user-invocable: true
metadata: {"openclaw":{"emoji":"🏋️","os":["darwin","linux","win32"],"requires":{"bins":["python"]},"primaryEnv":"COMFYUI_PATH"}}
---
# ComfyUI LoRA Training
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-prompt-engineer
description: Craft model-specific prompts optimized for the target checkpoint and identity method. Handles FLUX, SDXL, SD1.5, and Wan video models with proper syntax, quality tags, and negative prompts. Use when generating or refining prompts for ComfyUI workflows.
user-invocable: true
metadata: {"openclaw":{"emoji":"✍️","os":["darwin","linux","win32"]}}
---
# ComfyUI Prompt Engineer
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-research
description: Research latest ComfyUI models, techniques, and community discoveries. Monitors YouTube channels, GitHub repos, and HuggingFace. Updates reference files with timestamped findings and flags stale information. Invoke with /research comfyui or automatically at session start for staleness checks.
user-invocable: true
metadata: {"openclaw":{"emoji":"🔬","os":["darwin","linux","win32"]}}
---
# ComfyUI Research Skill
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-troubleshooter
description: Diagnose ComfyUI errors, workflow failures, and quality issues. Suggests fixes based on error patterns, missing dependencies, and community-known workarounds. Use when ComfyUI workflows fail or produce unexpected results.
user-invocable: true
metadata: {"openclaw":{"emoji":"🔍","os":["darwin","linux","win32"]}}
---
# ComfyUI Troubleshooter
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-video-pipeline
description: Generate videos using ComfyUI with Wan 2.2, FramePack, or AnimateDiff. Handles image-to-video, text-to-video, talking heads, and motion-controlled animation. Use when creating any video content from character images or text descriptions.
user-invocable: true
metadata: {"openclaw":{"emoji":"🎬","os":["darwin","linux","win32"],"requires":{"anyBins":["curl","wget"]},"primaryEnv":"COMFYUI_URL"}}
---
# ComfyUI Video Pipeline
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-voice-pipeline
description: Generate character voices using TTS, voice cloning, and lip-sync tools. Supports Chatterbox, F5-TTS, TTS Audio Suite, RVC, and ElevenLabs. Use when creating speech audio for characters or syncing audio to video.
user-invocable: true
metadata: {"openclaw":{"emoji":"🎙️","os":["darwin","linux","win32"],"requires":{"anyBins":["curl","wget"]},"primaryEnv":"COMFYUI_URL"}}
---
# ComfyUI Voice Pipeline
+2
View File
@@ -1,6 +1,8 @@
---
name: comfyui-workflow-builder
description: Generate ComfyUI workflow JSON from natural language descriptions. Validates against installed models/nodes before output. Use when building custom ComfyUI workflows from scratch or modifying existing ones.
user-invocable: true
metadata: {"openclaw":{"emoji":"🔧","os":["darwin","linux","win32"],"requires":{"anyBins":["curl","wget"]},"primaryEnv":"COMFYUI_URL"}}
---
# ComfyUI Workflow Builder
+2
View File
@@ -1,6 +1,8 @@
---
name: project-manager
description: Manage video production projects including character profiles, project manifests, workflow history, and asset tracking. Use when creating new projects, managing characters, or tracking production state.
user-invocable: true
metadata: {"openclaw":{"emoji":"📋","os":["darwin","linux","win32"]}}
---
# Project Manager
+2
View File
@@ -1,6 +1,8 @@
---
name: video-assembly
description: Assemble final video from generated clips, audio, and assets using FFmpeg or Remotion. Handles concatenation, audio mixing, transitions, titles, and export. Use when combining multiple production outputs into a final deliverable.
user-invocable: true
metadata: {"openclaw":{"emoji":"🎞️","os":["darwin","linux","win32"],"requires":{"bins":["ffmpeg"]}}}
---
# Video Assembly
+2
View File
@@ -1,6 +1,8 @@
---
name: video-publisher
description: Publish assembled videos to YouTube and other platforms. Orchestrates existing youtube-uploader, youtube-strategy, and youtube-plan-new-video skills. Use when ready to publish or plan distribution for completed videos.
user-invocable: true
metadata: {"openclaw":{"emoji":"📤","os":["darwin","linux","win32"]}}
---
# Video Publisher