sync: update baseball-trend, kiss-cam, language-swap, podcast from source main

- language-swap: AGNT-689 dub allowlist (ElevenLabs 32 + Minimax 8), zh normalization, extra-language + voice-preservation failure modes
- baseball-trend / kiss-cam / podcast: kling-v3-omni no-seed corrective-retry guidance; podcast jargon-heavy native audio handling + TTS fallback
This commit is contained in:
jiajunpika
2026-06-04 13:12:27 -07:00
parent 82247fba41
commit 7b370d60c6
5 changed files with 74 additions and 77 deletions
+7 -7
View File
@@ -20,7 +20,7 @@ Fixed-recipe skill — the prompts below are calibrated. Substitute the username
Before any paid MCP call, call `mcp__claude_ai_pika__identity_balance({verbose: true})` once. Surface the current balance, recent burn rate, and remaining runway, then gate the run with this exact message:
> Estimated cost: about 3,000-5,500 credits (~$30-$55) for the GPT-image-2 broadcast still, one or two Kling v3-omni pro 15s renders (includes one Step 2 re-render), and post-flight analyze_media QA. This exceeds $5, so Reply `proceed` to continue or `cancel` to stop.
> Estimated cost: about 3,000-5,500 credits (~$30-$55) for the GPT-image-2 broadcast still, one or two Kling v3-omni pro 15s renders (includes one Step 2 corrective retry with a changed payload), and post-flight analyze_media QA. This exceeds $5, so Reply `proceed` to continue or `cancel` to stop.
Do not call any paid MCP tool until the user replies `proceed`. If the user replies `cancel`, stop without generating. This is the only yes/no gate; after `proceed`, the pipeline runs end-to-end.
@@ -180,7 +180,7 @@ Constraints: Preserve identity strongly. Keep him seated behind home plate throu
Save the returned video URL as `state.broadcast_video_url`. If generation completes asynchronously, follow the MCP tool's returned status handle until the video reaches a terminal state.
Step 2 Kling video generation gets at most 2 total attempts (initial render + one targeted re-render for drift, scorebug/chyron movement, or announcer mispronunciation). Track `state.step2_attempt_count`. After either cap is exhausted, stop and ask for a better reference photo or permission to deliver the best attempt; include the best still/video URL and the failing check.
Step 2 Kling video generation gets at most 2 total attempts (initial render + one corrective retry for drift, scorebug/chyron movement, or announcer mispronunciation). kling-v3-omni has no seed, and identical Kling payloads can resolve to the same job/asset. Do not submit an identical Kling payload just to seek variation. Before the corrective retry, materially change the payload by using an updated `state.broadcast_still_url`, restoring missing strict params / negative_prompt entries, or shortening / clarifying the A/B announcer block. Track `state.step2_attempt_count`. After either cap is exhausted, stop and ask for a better reference photo or permission to deliver the best attempt; include the best still/video URL and the failing check.
### Step 3 — Deliver
@@ -204,7 +204,7 @@ Return JSON only: {
Check that the chyron still reads the exact username, the subject's identity stays stable throughout, the scorebug remains stable, the final clip has no black frames or wrong-sport shots, and the audio contains two distinct male announcer voices — not just one narrator.
```
- If `announcer_count < 2`, treat the result as at least `degraded`, include `audio_warning`, and re-roll Step 2 within the Step 2 retry budget instead of declaring success.
- If `announcer_count < 2`, treat the result as at least `degraded`, include `audio_warning`, and use the one Step 2 corrective retry only after changing the A/B audio payload. Do not submit an identical Kling payload.
- If `verdict` is `clean`, return the still URL and final video URL normally.
- If `verdict` is `degraded`, return the URLs plus the `quality_warning` and `audio_warning` so the user can review before publishing.
- If `verdict` is `catastrophic`, do not call the run complete; surface the verdict and `re_roll_suggestion` instead of declaring success.
@@ -229,7 +229,7 @@ The output-side gate is unavoidable for this trend regardless of subject, so See
**Kling caveat — recognizable celebrities are blocked too.** Kling has its own content-moderation gate that fires on celebrity references. A celebrity-reference prompt plus matching broadcast chyron can fail at submit-time with `task_status: failed, task_status_msg: "Failure to pass the risk control system"`. This is correct behavior — the trend illusion only works with a non-public-figure reference where the chyron name + face are coherent. If a user supplies a celebrity photo, surface the gate to them and ask for a non-celebrity reference instead.
**Kling trade-offs**: 2500-char `prompt` cap (recipe above is pre-trimmed), no `seed` param (re-rolls are non-reproducible — to re-roll just call again).
**Kling trade-offs**: 2500-char `prompt` cap (recipe above is pre-trimmed). kling-v3-omni has no seed; identical Kling payloads can collapse to the same job/asset, so a corrective retry must materially change the first-frame still, prompt, negative_prompt, or audio wording. Do not submit an identical Kling payload for variation.
## Runtime expectations
@@ -248,9 +248,9 @@ Typical run time is 4-7 minutes:
|---|---|---|
| Chyron pops in mid-clip (~45s flash) | Chyron not baked into the still | Re-run Step 1 within the Step 1 retry budget; verify chyron is visible in `state.broadcast_still_url` before Step 2 |
| Scorebug animates / morphs mid-clip | `prompt_adherence` not `strict`, or `negative_prompt` was trimmed | Restore strict adherence and the full negative_prompt |
| Identity drift late in the clip (face changes after ~10s) | Reference image too small / Kling losing the face | Re-run Step 2 within the Step 2 retry budget; if drift persists, re-run Step 1 with a tighter face crop on the still only if Step 1 budget remains |
| Only one announcer voice is heard | Kling collapsed the A/B commentary into one native narrator | Re-run Step 2 within the Step 2 retry budget; after the cap is exhausted, surface the audio warning and ask whether to deliver the best attempt |
| Username mispronounced by announcers | Native audio is one take | Re-run Step 2 within the Step 2 retry budget |
| Identity drift late in the clip (face changes after ~10s) | Reference image too small / Kling losing the face | Use the one Step 2 corrective retry only after materially changing the payload, usually by re-running Step 1 with a tighter face crop on the still if Step 1 budget remains |
| Only one announcer voice is heard | Kling collapsed the A/B commentary into one native narrator | Shorten or clarify the A/B announcer lines before the one Step 2 corrective retry; after the cap is exhausted, surface the audio warning and ask whether to deliver the best attempt |
| Username mispronounced by announcers | Native audio is one take | Add a pronunciation hint or shorten the announcer line before the one Step 2 corrective retry; otherwise surface the audio warning |
| OpenAI `moderation_blocked` on Step 1 | `gpt-image-2` safety gate on real person + ESPN-branded live feed + real MLB team context | Try `seedream` once with the same reference image and prompt while Step 1 budget remains. If it also blocks, tell the user: "OpenAI declined this image. Try using a fresh AI-generated headshot instead of a personal photo, or pick a non-MLB sport variant." |
| Seedance `partner_validation_failed` 422 | Tried Seedance instead of Kling | Use Kling only — see engine-choice section above |
| Kling `task_status: failed` with `task_status_msg: "Failure to pass the risk control system"` | Reference photo is a recognizable celebrity / public figure | Ask the user for a non-celebrity reference. Kling correctly blocks impersonation patterns (celebrity face + fake-event chyron) |
+8 -8
View File
@@ -22,7 +22,7 @@ pika MCP available in the host. Tool name prefix varies by mount point — use w
Before any paid MCP call, call `mcp__claude_ai_pika__identity_balance({verbose: true})` once. Surface the current balance, recent burn rate, and remaining runway, then gate the run with this exact message:
> Estimated cost: about 1,500-3,500 credits (~$15-$35) for the GPT-image-2 Jumbotron still, one or two Kling v3-omni pro 15s renders (includes one Step 2 re-render), and post-flight analyze_media QA. This exceeds $5, so Reply `proceed` to continue or `cancel` to stop.
> Estimated cost: about 1,500-3,500 credits (~$15-$35) for the GPT-image-2 Jumbotron still, one or two Kling v3-omni pro 15s renders (includes one Step 2 corrective retry with a changed payload), and post-flight analyze_media QA. This exceeds $5, so Reply `proceed` to continue or `cancel` to stop.
Do not call any paid MCP tool until the user replies `proceed`. If the user replies `cancel`, stop without generating. This is the only yes/no gate; after `proceed`, the pipeline runs end-to-end.
@@ -185,9 +185,9 @@ The working framing keeps the style-preservation lock but specifies subtle, rest
Save the returned video URL as `state.kisscam_video_url`. If generation completes asynchronously, follow the MCP tool's returned status handle. Client-layer timeouts can create an orphaned upstream task when no `task_id` reaches the agent; do not submit a duplicate. Surface the timeout and the orphaned upstream task risk to the user, then wait for a recoverable task handle or explicit operator confirmation before any rerun.
Step 2 Kling video generation gets at most 2 total attempts (initial render + one targeted re-render for text/scoreboard animation, identity drift, kiss timing, PA timing, or lip-sync artifacts). Track `state.step2_attempt_count`.
Step 2 Kling video generation gets at most 2 total attempts (initial render + one corrective retry for text/scoreboard animation, identity drift, kiss timing, PA timing, or lip-sync artifacts). kling-v3-omni has no seed, and identical Kling payloads can resolve to the same job/asset. Do not submit an identical Kling payload just to seek variation. Before the corrective retry, materially change the payload by using an updated `state.kisscam_still_url`, restoring missing strict params / negative_prompt entries, or changing the PA audio wording / timing. Track `state.step2_attempt_count`.
**On failure**: re-run kling within the Step 2 cap — don't switch video engines. Seedance's two-stage likeness gate (same as the `baseball-trend` sibling) makes it unusable here. If the still itself is the issue (use the Step 1 self-check criteria — text spelling, scoreboard, likeness — to decide), re-run Step 1 only if Step 1 budget remains. After either cap is exhausted, stop and ask for safer references or permission to deliver the best attempt; include the best still/video URL and the failing check.
**On failure**: use the Step 2 corrective retry only after changing the Kling payload — don't switch video engines. Seedance's two-stage likeness gate (same as the `baseball-trend` sibling) makes it unusable here. If the still itself is the issue (use the Step 1 self-check criteria — text spelling, scoreboard, likeness — to decide), re-run Step 1 only if Step 1 budget remains and use the new still as the changed first frame. After either cap is exhausted, stop and ask for safer references or permission to deliver the best attempt; include the best still/video URL and the failing check.
## Step 3 — Deliver
@@ -261,7 +261,7 @@ Don't edit these without a re-validation pass — they're empirical behavior dep
**Step 2 — Kling, no Seedance.** Seedance has a two-stage `partner_validation_failed` 422 gate (same as the `baseball-trend` sibling skill): an input-side gate that rejects references with recognizable real people, and an output-side gate that rejects AFTER generation if the produced clip contains recognizable-looking faces. Every Kiss Cam shot has a packed-arena crowd full of faces, so the output-side gate is unavoidable here. Kling is the only engine that lands this recipe.
**Kling trade-offs**: 2500-char `prompt` cap (recipe above is pre-trimmed to ~2400 chars; re-inflating it can trigger prompt-length errors), no `seed` param (re-rolls are non-reproducible — to re-roll just call again).
**Kling trade-offs**: 2500-char `prompt` cap (recipe above is pre-trimmed to ~2400 chars; re-inflating it can trigger prompt-length errors). kling-v3-omni has no seed; identical Kling payloads can collapse to the same job/asset, so a corrective retry must materially change the first-frame still, prompt, negative_prompt, or PA audio wording. Do not submit an identical Kling payload for variation.
## Failure cheat sheet
@@ -269,11 +269,11 @@ Don't edit these without a re-validation pass — they're empirical behavior dep
|---|---|
| `moderation_blocked` on Step 1 | gpt-image-2 safety gate (often female/female subject pairings + "kiss cam" wording). Re-roll the same call within the Step 1 retry budget; do NOT swap providers — alternatives produce softer likeness and softer LED detail |
| Kling prompt error: prompt > 2500 chars | Re-inflated audio or aesthetic section in the Kling prompt. Cut from the audio or aesthetic block; never from the animation timeline |
| Scoreboard, "Kiss Cam" text, or graphics animate mid-clip | `prompt_adherence` not set to `strict`, or `negative_prompt` missing entries like "scoreboard changing" / "Kiss Cam text changing". Verify both params; re-run Step 2 within the Step 2 retry budget |
| Subject identity drifts after ~8s | Reference still face crop too small — not enough facial pixels for Kling to lock. Re-roll Step 1 with a tighter face crop on the subjects only if Step 1 budget remains |
| Scoreboard, "Kiss Cam" text, or graphics animate mid-clip | `prompt_adherence` not set to `strict`, or `negative_prompt` missing entries like "scoreboard changing" / "Kiss Cam text changing". Restore both params before the one Step 2 corrective retry so the payload differs |
| Subject identity drifts after ~8s | Reference still face crop too small — not enough facial pixels for Kling to lock. Re-roll Step 1 with a tighter face crop on the subjects only if Step 1 budget remains, then use the changed still for the Step 2 corrective retry |
| Subject gets redrawn in a different style (photoreal → illustrated, or vice versa) | Style-preservation lock weakened, or a specific style label (Pixar / anime / etc.) crept into either prompt. Restore the "preserve exact likeness AND visual style" anchor; remove any style label |
| PA announcer mispronounces a word or misses the kiss beat | Native audio is one take per Kling generation. Re-run Step 2 within the Step 2 retry budget — no prompt-level fix |
| One of the on-screen subjects lip-syncs / mouths the PA announcer's dialogue | Kling defaults to attributing any quoted dialogue on the audio track to a visible face in frame — without an explicit off-screen anchor, it picks a subject and animates their mouth to the words. Verify the audio block is framed as `Audio (non-diegetic / OFF-SCREEN only — Subject A and Subject B stay SILENT throughout, mouths closed...)` and the `negative_prompt` contains `subjects lip-syncing PA announcer dialogue, characters mouthing the announcer lines, on-screen lip-sync`; re-run Step 2 within the Step 2 retry budget |
| PA announcer mispronounces a word or misses the kiss beat | Native audio is one take per Kling generation. Add a pronunciation hint, shorten the PA line, or adjust the audio timing before the one Step 2 corrective retry |
| One of the on-screen subjects lip-syncs / mouths the PA announcer's dialogue | Kling defaults to attributing any quoted dialogue on the audio track to a visible face in frame — without an explicit off-screen anchor, it picks a subject and animates their mouth to the words. Verify the audio block is framed as `Audio (non-diegetic / OFF-SCREEN only — Subject A and Subject B stay SILENT throughout, mouths closed...)` and the `negative_prompt` contains `subjects lip-syncing PA announcer dialogue, characters mouthing the announcer lines, on-screen lip-sync`; restore those anchors before the one Step 2 corrective retry |
| Subjects mug at camera / over-act / theatrical expressions / cartoon reactions | Animation block prescribes a loaded list of simultaneous micro-expressions, or timeline beats use exaggerated descriptors ("huge smile", "shy excited wiggle", "eyes widen"). Restore the `subtle, restrained, true-to-life motion` framing in the animation block; strip emotion adjectives from the timeline; verify `exaggerated acting / theatrical expressions / over-acting / mugging at camera / cartoon reactions` are in the `negative_prompt` |
| Kiss lands on cheek / forehead / head instead of lips | Timeline beat softened away from `gentle kiss on the lips`. Restore the verbatim lip-kiss line in the 7-11s beat; the trend is a lip kiss, not a peck on the head |
| Seedance attempted instead of Kling | Wrong engine chosen. Switch to `kling` — Seedance's two-stage likeness gate makes it unusable here (see "Engine choice") |
+3 -2
View File
@@ -26,7 +26,7 @@ The dubbing worker does the heavy lifting in a single call: it transcribes, tran
## Behavior defaults
- **Target language**: required via `--to <language>`. Prefer language codes: `es`, `fr`, `ja`, `de`, `pt-BR`, `zh-Hans`. The dubbing worker accepts ISO/BCP-47-like tags and normalizes script subtags before calling ElevenLabs (for example `zh-Hans``zh`; `zh-Hant-TW``zh-TW`).
- **Target language**: required via `--to <language>`. Prefer language codes: `es`, `fr`, `ja`, `de`, `pt-BR`, `zh-Hans`. The dubbing worker accepts ISO/BCP-47-like tags and normalizes script/region subtags before calling ElevenLabs (for example `zh-Hans``zh`; `zh-Hant-TW``zh`).
- **Lipsync**: ON by default — re-matches the speaker's mouth to the translated audio (fal sync-lipsync; the full-video lip-matcher, distinct from the portrait-image animator). Pass `--no-lipsync` to skip it when the source has no on-camera face or to avoid the meaningful cost (~$4/min on the sync-2-pro tier).
- **BGM / background music**: kept by default — the dub lays the translated voice over the original music / SFX bed. Pass `--no-bgm` for a translate-only output: the worker drops the original music and keeps only the translated speech (`drop_background_audio=true`).
- **Language coverage**: if language support is questioned or a language-related upstream error occurs, consult `references/language-coverage.md`. Do not proactively surface provider-specific language-list details in normal user replies.
@@ -112,9 +112,10 @@ Reply with `final_video_url` + the translated transcript (from `dub_transcript_s
| Class | Trigger | Mitigation | Fallback |
|---|---|---|---|
| Source URL not worker-fetchable | `mcp__plugin_pika_pika__dub_video` returns 403 / 4xx, hotlink / UA-gated fetch failure, or "Access Denied" for a public HTTPS URL | Download source bytes in the client/host environment, `mcp__claude_ai_pika__upload_asset` them to Pika, replace `video_url` with the Pika CDN URL, then retry Step 1 once | If local download also fails, ask the user to upload the file or provide a different URL |
| Extra target language | Target is Cantonese (`yue` / `cantonese` / `zh-HK`), Thai, Hebrew, Persian, Slovenian, Catalan, Norwegian Nynorsk, or Afrikaans | Supported — call `mcp__plugin_pika_pika__dub_video` with the target as usual; the original speaker's voice is kept | Background music isn't preserved for these languages (dubbed speech only) |
| Dub call fails (not fetchability) | `mcp__plugin_pika_pika__dub_video` errors for another reason — unsupported target language, provider/worker 5xx, `status: failed` from `mcp__plugin_pika_pika__task_status` | Surface the error to the user; if the message points at the language, check `references/language-coverage.md` and suggest a supported tag; otherwise suggest a retry. There is no manual chain to fall back to — dub is the single path | None — return the error, do not silently produce a non-dubbed video |
| Dub returns no speech | Silent video — nothing to translate | Surface to user: "no detectable speech in video — nothing to translate" | None |
| Voice not preserved | `mcp__plugin_pika_pika__dub_video` cannot isolate/clone the speaker (very short or noisy source) | Surface to the user that the original voice could not be preserved and the dub uses a provider default voice | Provider default voice |
| Original voice can't be kept | For the languages above, the source is too short or noisy to keep the original speaker's voice | Surface the error and ask the user for a cleaner / longer source clip | None — the dub fails rather than using a different voice |
| Lipsync step fails | `mcp__claude_ai_pika__edit_lipsync` errors (no clear face track, provider 4xx) | Fall back through `variant` tiers (v2-pro → sync-3 → v2); if all fail, return the dubbed video without lip-matching and tell the user | Audio-replaced video, no lip-match |
| Captions wrong language | Step 3 auto-transcription mis-detects language | Pass explicit `language` tag; if `dub_subtitles` exists, use `caption_mode="manual"` with it instead of auto | Manual `subtitles[]` |
@@ -1,6 +1,6 @@
# Language Coverage Reference
Last verified: 2026-06-02 PST
Last verified: 2026-06-04 PST
## Purpose
@@ -10,101 +10,81 @@ Do not proactively expose provider-specific language-list details in normal user
## Boundary
The fast-path dub accepts a target language from the user and forwards it upstream as a language code. The worker does not own a hardcoded allowlist.
The `dub_video` worker **owns a hardcoded, empirically-verified allowlist** (AGNT-689). It is no longer a blind pass-through. Two routing paths:
The current upstream dubbing API reference documents `target_lang` and `source_lang` as valid `iso639-1` or `iso639-3` language codes, but does not publish a complete enum of accepted dubbing languages.
- **ElevenLabs dubbing** handles **32 languages** — the codes ElevenLabs `eleven_dubbing` actually accepts, verified 2026-06-04 by submitting every code through the dubbing proxy. A target outside this set (and outside the Minimax set below) is rejected **before billing** with the supported list.
- **Minimax branch (voice-cloned)** handles **8 languages ElevenLabs cannot dub**: Cantonese, Thai, Hebrew, Persian, Slovenian, Catalan, Norwegian Nynorsk, Afrikaans. For these the worker clones the source speaker's voice and synthesizes with Minimax. Trade-offs on this path: one cloned voice for the whole clip, and the original background-music bed is replaced by the dubbed speech.
The language list below is a model-capability reference for speech generation / fallback behavior. It is not a guaranteed dubbing API allowlist.
**Mandarin gotcha:** ElevenLabs accepts only `zh`. The ISO-639-3 code `cmn` is rejected upstream; the worker auto-maps `cmn`/`zho`/`chi` to `zh`.
The upstream dubbing product docs also advertise 90+ languages for a newer product path while noting that the newer API is not live yet. Do not use that 90+ claim as a guarantee for the current MCP worker path unless the API contract changes.
Source note: verified from upstream help center language-support docs and upstream dubbing API/product docs on the date above. Keep provider-specific names and links out of normal user-facing replies unless the user explicitly asks for the external source.
Source note: the 32-language ElevenLabs set and the `cmn`-rejected behavior were verified empirically against the dubbing proxy on the date above. The 8-language Minimax set is Minimax Speech-02's `language_boost` coverage minus the ElevenLabs 32. Keep provider-specific names and links out of normal user-facing replies unless the user explicitly asks for the external source.
## 74-Language Model Reference
## Supported languages
The Help Center lists 74 languages:
### ElevenLabs dubbing — 32 languages (verified)
| Language | ISO-639 code shown by the source |
| Language | Code sent upstream |
| --- | --- |
| Afrikaans | `afr` |
| Arabic | `ara` |
| Armenian | `hye` |
| Assamese | `asm` |
| Azerbaijani | `aze` |
| Belarusian | `bel` |
| Bengali | `ben` |
| Bosnian | `bos` |
| Bulgarian | `bul` |
| Catalan | `cat` |
| Cebuano | `ceb` |
| Chichewa | `nya` |
| Croatian | `hrv` |
| Czech | `ces` |
| Danish | `dan` |
| Dutch | `nld` |
| English | `eng` |
| Estonian | `est` |
| Filipino | `fil` |
| Finnish | `fin` |
| French | `fra` |
| Galician | `glg` |
| Georgian | `kat` |
| German | `deu` |
| Greek | `ell` |
| Gujarati | `guj` |
| Hausa | `hau` |
| Hebrew | `heb` |
| Hindi | `hin` |
| Hungarian | `hun` |
| Icelandic | `isl` |
| Indonesian | `ind` |
| Irish | `gle` |
| Italian | `ita` |
| Japanese | `jpn` |
| Javanese | `jav` |
| Kannada | `kan` |
| Kazakh | `kaz` |
| Kirghiz | `kir` |
| Korean | `kor` |
| Latvian | `lav` |
| Lingala | `lin` |
| Lithuanian | `lit` |
| Luxembourgish | `ltz` |
| Macedonian | `mkd` |
| Malay | `msa` |
| Malayalam | `mal` |
| Mandarin Chinese | `cmn` |
| Marathi | `mar` |
| Nepali | `nep` |
| Norwegian | `nor` |
| Pashto | `pus` |
| Persian | `fas` |
| Mandarin Chinese | `zh` (NOT `cmn`) |
| Norwegian (Bokmål) | `nor` |
| Polish | `pol` |
| Portuguese | `por` |
| Punjabi | `pan` |
| Romanian | `ron` |
| Russian | `rus` |
| Serbian | `srp` |
| Sindhi | `snd` |
| Slovak | `slk` |
| Slovenian | `slv` |
| Somali | `som` |
| Spanish | `spa` |
| Swahili | `swa` |
| Swedish | `swe` |
| Tamil | `tam` |
| Telugu | `tel` |
| Thai | `tha` |
| Turkish | `tur` |
| Ukrainian | `ukr` |
| Urdu | `urd` |
| Vietnamese | `vie` |
| Welsh | `cym` |
ISO-639-1 spellings (`es`, `fr`, `ja`, `pt-BR`, …) are accepted and forwarded; region subtags are tolerated and script subtags are dropped.
### Minimax voice-clone branch — 8 languages
These are NOT supported by ElevenLabs dubbing; the worker routes them to Minimax with a cloned source voice (AGNT-689).
| Language | Accepted target codes |
| --- | --- |
| Cantonese | `yue`, `cantonese`, `zh-HK` |
| Thai | `tha`, `th` |
| Hebrew | `heb`, `he` |
| Persian (Farsi) | `fas`, `fa` |
| Slovenian | `slv`, `sl` |
| Catalan | `cat`, `ca` |
| Norwegian Nynorsk | `nno`, `nn` (Bokmål `no`/`nb` stays on ElevenLabs) |
| Afrikaans | `afr`, `af` |
## Not supported
Many languages that appear in ElevenLabs' general *speech-generation* help-center list (e.g. Welsh, Icelandic, Serbian, Swahili, Urdu, Lithuanian, Irish, and others) are **not** accepted by the dubbing API and are not in the Minimax set. A model-capability list is **not** a dubbing allowlist — passing such a code now fails fast with the supported list rather than erroring opaquely mid-pipeline.
## Agent Guidance
When the user asks "how many languages does Language Swap support?", answer carefully:
- For the current fast-path dub worker: upstream is the source of truth; the MCP worker accepts ISO-style `target_language` values and forwards them as `target_lang`.
- For a named reference list: cite the 74-language list above, and explicitly say it is not a published dubbing API enum.
- The fast-path dub supports **40 languages total**: 32 via ElevenLabs + 8 via the Minimax voice-clone branch.
- The two paths are an implementation detail — do not surface provider names in normal replies. If a requested language is unsupported, state that plainly and, when helpful, point to the closest supported option.
- For the newer 90+ language product claim: mention it only with the API-not-live caveat from the upstream dubbing overview.
+22 -6
View File
@@ -32,7 +32,7 @@ argument-hint: <url-or-topic> [bg_img=] [host_a_img=] [host_b_img=] [voice_a=] [
Before any paid MCP call, call `mcp__claude_ai_pika__identity_balance({verbose: true})` once. Surface the current balance, recent burn rate, and remaining runway, then gate the run with this exact message:
> Estimated cost: about 6,000-9,000 credits (~$60-$90) for four Kling v3-omni pro 15s acts, optional missing-asset image generation, one act re-render, concat, and post-flight analyze_media QA. This exceeds $5, so Reply `proceed` to continue or `cancel` to stop.
> Estimated cost: about 6,000-9,000 credits (~$60-$90) for four Kling v3-omni pro 15s acts, optional missing-asset image generation, one act corrective retry, concat, and post-flight analyze_media QA. This exceeds $5, so Reply `proceed` to continue or `cancel` to stop.
Do not call any paid MCP tool until the user replies `proceed`. If the user replies `cancel`, stop without generating. This is the only yes/no gate; after `proceed`, render the four acts and return the URL.
@@ -153,6 +153,18 @@ Use this section when the URL, topic, product name, host name, quote, or require
- **Captions / font fallback**: this skill skips `add_captions` by default because podcast dialogue mistranscribes jargon. If the user explicitly asks for captions anyway, pass manual `subtitles[]` from the authored script rather than auto-transcribing, set `font: "noto-cjk"` for Chinese / Japanese / Korean, and preserve the original non-Latin text exactly.
- **QA**: before delivery, include a one-line note in the final verdict when non-Latin text was present: whether original characters were preserved in the script and whether the selected voices were language-matched or best-effort.
### Jargon-heavy native audio handling
Use this section when the URL, topic, product name, quote, or required script detail is domain-heavy or jargon-heavy, especially finance, law, medicine, AI/crypto, acronyms, coined terms, unusual product names, long numbers, or multi-word compounds. Kling-omni native TTS can garble coined terms in the spoken native audio, not just in auto-captions, so reduce the risk before the paid Kling calls.
- **Risk scan**: mark the run jargon-heavy when a line would contain several domain-specific terms, acronyms, long numbers, or nested / multi-clause economist-style phrasing. Treat examples like stablecoins, unbundle, and reassure as load-bearing terms that must remain intelligible.
- **Script rewrite**: preserve the canonical term in the script state and final recap, but shorten the spoken line around it. Use short lines, split long clauses, keep at most one or two high-risk terms per host line, and avoid nested clauses that make Kling infer similar-sounding filler words.
- **Pronunciation aid**: create a small per-act pronunciation aid for high-risk terms, for example `stablecoins = STAY-bul coins`, `unbundle = un-BUN-dul`, `reassure = ree-uh-SHOOR`. The pronunciation aid is prompt metadata only; do not replace the canonical term in the dialogue, captions, facts, or final recap.
- **Known-hard-term correction**: if a term is known to drift, or post-flight localizes a near-neighbor mishearing, do not retry the same standalone hard word. Keep the canonical term in act metadata (`canonical_terms: ["unbundle"]`), then make the corrected voice-token line a short context phrase with a speakable cue / micro-pause, for example `we un-bundle payments`. Add the observed wrong form as a negative pronunciation guard in metadata, for example `unbundle = un-BUN-dul, not un-bumble`.
- **Known-hard publishable fallback**: before the paid Kling calls, mark observed hard terms such as `unbundle` as known-hard when prior E2E or the current post-flight findings show drift like `unbundle` -> `unbubble`, `un-bumble`, or `unbundable`. Record `canonical_terms` and `observed_wrong_forms`. Do not render known-hard spoken terms with Kling native audio. Instead, generate the affected visual act or final video with no spoken known-hard term in the native voice-token lines, then generate controlled narration with `generate_speech(text=<spoken-only script>, provider: "minimax-tts", minimax_model: "speech-2.8-hd", language: "en")` using the canonical spelling (`unbundle`, not the hyphen cue). The spoken-only script must preserve the authored host-turn order but contain no HOST_A/HOST_B labels, no metadata, no pronunciation aids, and no observed wrong-form spellings. Run pre-replacement QA on the TTS audio with both `transcribe_audio(audio=<tts_audio_url>)` and `analyze_media(media=<tts_audio_url>)`; every required `canonical_terms` entry must appear in the `transcribe_audio` text and must appear in the `analyze_media` observed transcript. Reject the TTS if a canonical term is missing, replaced by any observed wrong form, or replaced by a new near-neighbor. Then call `edit_audio_replace(video_url=<final_mp4>, audio_url=<tts_audio_url>, duration_policy: "video")` to discard the native audio and replace it with the controlled TTS track. Run post-replacement QA on the replaced MP4 with both `transcribe_audio(audio=<replaced_mp4_url>)` and `analyze_media(media=<replaced_mp4_url>)`; every required `canonical_terms` entry must appear in both final checks, and `analyze_media` must also verify host-turn order and acceptable audio/video sync. If any final check hears a wrong form, misses a canonical term, or says the fallback collapses the podcast turn structure, return `not publish-ready`; if both checks pass, the fallback output is publish-ready even though it uses controlled TTS instead of Kling native speech.
- **Numbers and acronyms**: write spoken-friendly forms when they matter: `$175B` becomes `one hundred seventy-five billion dollars`; `API` becomes `A P I` unless the brand normally says it as a word.
- **Act prompt**: include the per-act pronunciation aid near the voice-token lines with an instruction that it is not dialogue and should not be spoken verbatim. Ask for clear pronunciation of the canonical terms and no improvising similar words.
### 5. Write script
Write 4 acts × 2 lines (HOST_A / HOST_B). Each line ~1012s of spoken dialogue.
@@ -165,6 +177,7 @@ Write 4 acts × 2 lines (HOST_A / HOST_B). Each line ~1012s of spoken dialogu
- Real reactions, not generic praise
- Reference at least one actual feature name, price, claim, or quote
- Natural ending — no forced "bye!"
- Apply Jargon-heavy native audio handling before finalizing the act lines when the topic is domain-heavy, jargon-heavy, or uses coined terms.
Acts: Hook → Feature deep-dive → The Turn → Verdict
(In topic mode the analogue: Hook → Substance → The Pivot → Verdict.)
@@ -173,8 +186,9 @@ Acts: Hook → Feature deep-dive → The Turn → Verdict
Delegate to a subagent with all resolved assets and the script. The subagent runs acts 1→2→3→4 sequentially — do NOT parallelize. The subagent must follow the Long-running task_status polling contract above and relay every 60s progress line back to the parent while it is waiting on Kling, image generation, voice clone, or concat tasks.
Each act: one `generate_reference_video` call (`kling-v3-omni`, `duration=15`, `sound=true`, `quality_mode: "pro"`). Pass `reference_images=[bg_img, host_a_img, host_b_img]`, `voice_ids=[voice_a, voice_b]`, and `quality_mode: "pro"` on every act; this must pass `quality_mode: "pro"` because the cost gate quotes the pro-tier 15s act cost. Optional knob: `kling_model` to pin a specific kling family member if you need reproducibility across runs. Three shots:
Each normal act: one `generate_reference_video` call (`kling-v3-omni`, `duration=15`, `sound=true`, `quality_mode: "pro"`). Pass `reference_images=[bg_img, host_a_img, host_b_img]`, `voice_ids=[voice_a, voice_b]`, and `quality_mode: "pro"` on every normal act; this must pass `quality_mode: "pro"` because the cost gate quotes the pro-tier 15s act cost. Optional knob: `kling_model` to pin a specific kling family member if you need reproducibility across runs. Three shots:
- **Known-hard fallback exception**: for acts containing known-hard spoken terms, render the affected visual act with `sound=false` and no voice tokens for the affected lines. Do not include canonical known-hard terms in any `<<<voice_*>>>` native voice-token line; the controlled TTS replacement supplies those words later. For jargon-heavy but not known-hard acts, include the per-act pronunciation aid from Step 5 in the act prompt as non-dialogue metadata and preserve canonical terms in the quoted `<<<voice_*>>>` lines.
- Wide 5s: both hosts, no voice token
- MCU-A 5s: `<<<voice_1>>> '<HOST_A line>'`
- MCU-B 5s: `<<<voice_2>>> '<HOST_B line>'`
@@ -185,7 +199,7 @@ Emotional beats per act:
- Act 3: A firm, B surprised and reconsidering
- Act 4: A satisfied, B conceding
After act 4, subagent calls `edit_concat([act1, act2, act3, act4])`, relays any async polling progress to the parent, and returns the final video URL. Keep the four act URLs in state so the post-flight quality gate can spend at most one targeted act re-render without regenerating clean acts.
After act 4, subagent calls `edit_concat([act1, act2, act3, act4])`, relays any async polling progress to the parent, and returns the final video URL. Keep the four act URLs in state so the post-flight quality gate can spend at most one targeted act correction without regenerating clean acts.
### 7. Output
@@ -203,11 +217,13 @@ Return JSON only: {
"re_roll_suggestion": string | null
}
Check that Host A remains on the left, Host B remains on the right, product names / persona names / topic-specific terms are not visibly garbled, the four-act podcast structure is present, and there are no black frames or wrong-host shots.
Also check the spoken audio, not only captions or visible text: product names, persona names, topic-specific terms, coined terms, and jargon must be intelligible. Flag garbled or mispronounced native audio such as "stablecoins" sounding like "stable kinds", "unbundle" sounding like "un-bumble" or "unbundable", or "reassure" losing syllables.
```
- If `verdict` is `clean`, return the final URL and one-sentence verdict normally.
- If `verdict` is `degraded`, return the final URL plus the `quality_warning` so the user can review before publishing.
- If `verdict` is `catastrophic`, spend the one targeted act re-render budget only when the observations or `re_roll_suggestion` identifies a single bad act, wrong-host shot, black frame, or localized visual/audio failure. Re-render that act once with `quality_mode: "pro"`, re-concat the four acts, and run this analyze_media check one more time. Do not exceed the cost-gate range: if the second verdict is still `catastrophic`, or if the failure is not attributable to one act, do not call the podcast complete; surface the verdict and `re_roll_suggestion` instead of declaring success.
- If `verdict` is `clean`, first check whether the script state contains a known-hard spoken term. For known-hard terms, before accepting a clean verdict, run the Known-hard publishable fallback and its pre-replacement plus post-replacement `transcribe_audio` / `analyze_media` checks. Only return success after the replaced MP4 checks pass. If there are no known-hard spoken terms, return the final URL and one-sentence verdict normally.
- If `verdict` is `degraded` for ordinary non-blocking visual issues, return the final URL plus the `quality_warning` so the user can review before publishing.
- If `verdict` is `degraded` because spoken audio garbles jargon, coined terms, or other domain-specific terms, treat it as not publish-ready instead of silent success. For a known-hard term or observed hard-term drift, skip the native-audio act correction; do not spend the targeted act correction budget on another Kling native-audio prompt. Use the Known-hard publishable fallback directly: call `generate_speech(text=<spoken-only script>, provider: "minimax-tts", minimax_model: "speech-2.8-hd")`, run pre-replacement `transcribe_audio` and `analyze_media` on the TTS audio, then call `edit_audio_replace(video_url=<final_mp4>, audio_url=<tts_audio_url>, duration_policy: "video")` to discard the native audio and replace the final MP4 audio. Run post-replacement `transcribe_audio` and `analyze_media` on the replaced MP4. If those checks pass, return the replaced URL; if the fallback checks still report spoken jargon garble, or if the garble is not localized enough to correct or replace, do not call the podcast complete; return the URL with `not publish-ready`, affected terms, and the `quality_warning`. For non-known-hard garble only, if the observations or `re_roll_suggestion` localize the problem to one act, spend the one targeted act correction budget: materially change that act payload by shortening the affected line, splitting multi-clause wording, adding or refining the phonetic pronunciation aid, changing the shot prompt, reference image, voice ID, or other stage-specific input that caused the failure. Submit one corrected act with `quality_mode: "pro"`, re-concat the four acts, and run this analyze_media check one more time. If the second verdict still reports spoken jargon garble, do not call the podcast complete; return `not publish-ready`.
- If `verdict` is `catastrophic`, spend the one targeted act correction budget only when the observations or `re_roll_suggestion` identifies a single bad act, wrong-host shot, black frame, or localized visual/audio failure. kling-v3-omni has no seed, and identical act payloads can resolve to the same job/asset. Do not submit an identical Kling payload. Before retrying, materially change the bad act payload by adjusting the script line, shot prompt, reference image, voice ID, pronunciation aid, or other stage-specific input that caused the failure. Submit one corrected act with `quality_mode: "pro"`, re-concat the four acts, and run this analyze_media check one more time. Do not exceed the cost-gate range: if the second verdict is still `catastrophic`, or if the failure is not attributable to one act, do not call the podcast complete; surface the verdict and `re_roll_suggestion` instead of declaring success.
---