Merge pull request #229 from calesthio/codex/expressive-tts-governance

[codex] Add expressive TTS governance

.agents/skills/hyperframes-media/references/tts.md

@@ -65,6 +65,27 @@ node skills/hyperframes-media/scripts/heygen-tts.mjs --list   # public starfish
 | Offline, no API key, fast iteration                       | **Kokoro**                                          |
 | Non-English multilingual with deterministic phonemization | **Kokoro** (`ef_dora`, `jf_alpha`, `zf_xiaobei`, …) |
 
+## Expressive narration contract
+
+Before generating narration, write a compact voice-performance plan:
+
+- `performance_intent` - who the narrator is and how they should feel
+- `pacing_profile` - contemplative, conversational, energetic, technical, or custom
+- `energy_curve` - how the read changes across the piece
+- `pause_policy` - where silence should happen and why
+- section-level cues - `pace`, `energy`, `emphasis_words`, `pause_before_seconds`,
+  `pause_after_seconds`, and optional provider-ready text
+
+Do not rely on a vague instruction like "make it natural." Put the direction in
+the text or provider settings:
+
+- Use short sentences and purposeful punctuation.
+- Use `<break time="0.4s"/>` to `<break time="1.0s"/>` for important pauses when
+  the chosen provider supports SSML-style break tags.
+- Generate a sample from the most performance-sensitive section before batching.
+- If the sample sounds monotone, rushed, or ignores pauses, revise the plan or
+  provider settings before generating the rest.
+
 ## ffmpeg requirement
 
 HeyGen + ElevenLabs return mp3. The CLI transcodes to wav when `--output` ends in `.wav` (the default and what downstream `ffprobe` + Whisper expect). If you'd rather skip the transcode, pass `-o file.mp3`. Without `ffmpeg` on PATH, `.wav` output from the cloud providers fails — install ffmpeg or use `.mp3`.

.agents/skills/text-to-speech/SKILL.md

@@ -356,6 +356,25 @@ Rules:
 - Must have spaces before and after the tag
 - Self-closing tag format
 
+## Expressive Voice Direction
+
+For narration, create a short voice-performance plan before generating audio:
+
+- narrator persona and emotional intent
+- pacing profile
+- energy curve across the script
+- where pauses should land
+- words or phrases that need emphasis
+
+Use concrete cues, not generic instructions. "Warm but decisive; pause before
+the contrast; slow down on the final sentence" is useful. "Sound natural" is
+not.
+
+When the selected voice supports pauses, put the most important pauses directly
+in the text with break tags. Generate a sample from the most performance-heavy
+section first, and do not batch-generate the rest if the sample sounds flat,
+rushed, or ignores the intended breaks.
+
 ## Best Practices
 
 1. **Use `GET /v1/audio/voices`** to find compatible voices — not all voices from `GET /v2/voices` support Starfish TTS

.claude/skills/text-to-speech/SKILL.md

@@ -356,6 +356,25 @@ Rules:
 - Must have spaces before and after the tag
 - Self-closing tag format
 
+## Expressive Voice Direction
+
+For narration, create a short voice-performance plan before generating audio:
+
+- narrator persona and emotional intent
+- pacing profile
+- energy curve across the script
+- where pauses should land
+- words or phrases that need emphasis
+
+Use concrete cues, not generic instructions. "Warm but decisive; pause before
+the contrast; slow down on the final sentence" is useful. "Sound natural" is
+not.
+
+When the selected voice supports pauses, put the most important pauses directly
+in the text with break tags. Generate a sample from the most performance-heavy
+section first, and do not batch-generate the rest if the sample sounds flat,
+rushed, or ignores the intended breaks.
+
 ## Best Practices
 
 1. **Use `GET /v1/audio/voices`** to find compatible voices — not all voices from `GET /v2/voices` support Starfish TTS

pipeline_defs/animated-explainer.yaml

@@ -40,6 +40,7 @@ required_skills:
   - meta/checkpoint-protocol
   - meta/skill-creator
   - meta/animation-runtime-selector
+  - meta/voice-performance-director
 
 orchestration:
   mode: executive-producer
@@ -130,6 +131,7 @@ stages:
       - "Enhancement cue density: 1 per 8-10 seconds"
       - "Narrative arc: hook -> setup -> build -> climax -> landing"
       - Speaker directions present for TTS
+      - Voice performance plan includes concrete pacing, pause, emphasis, and sample cues
       - Script incorporates research data points and cited facts
     success_criteria:
       - Schema-valid script with enhancement_cues per section
@@ -186,6 +188,7 @@ stages:
     review_focus:
       - All asset files exist on disk
       - Narration covers all script sections
+      - Narration assets apply the approved voice performance plan and sample settings
       - Total cost within approved budget from proposal_packet
       - Image style consistency across all generated visuals
     success_criteria:

pipeline_defs/animation.yaml

@@ -40,6 +40,7 @@ required_skills:
   - meta/reviewer
   - meta/checkpoint-protocol
   - meta/animation-runtime-selector
+  - meta/voice-performance-director
 
 orchestration:
   mode: executive-producer
@@ -137,6 +138,7 @@ stages:
       - Script beats are concise and animation-friendly
       - On-screen text remains readable and restrained
       - Timing structure leaves room for visual holds
+      - Narration-led scripts include concrete voice performance cues
       - Word count within +/-10% of duration target from selected concept
       - Script incorporates research data points where relevant
     success_criteria:
@@ -196,6 +198,7 @@ stages:
       - Asset production path is explicit per scene
       - Reusable motifs and templates are prepared and referenced
       - Missing tool paths are surfaced honestly
+      - Narration assets apply the approved voice performance plan and sample settings
       - Total cost within approved budget from proposal_packet
       - "Layer 3 skills read for EVERY generation tool before writing prompts (check agent_skills field)"
       - "Clip duration maximized (prefer 10s over 5s) to reduce API calls and cost"

schemas/artifacts/asset_manifest.schema.json

@@ -33,7 +33,24 @@
           "generation_summary": { "type": "string", "description": "Brief summary of how the asset was generated or sourced" },
           "provider": { "type": "string", "description": "Provider name (e.g. pixabay, google_imagen)" },
           "license": { "type": "string", "description": "License type (e.g. Pixabay License, CC0)" },
-          "original_url": { "type": "string", "description": "Source URL if downloaded from a stock service" }
+          "original_url": { "type": "string", "description": "Source URL if downloaded from a stock service" },
+          "voice_performance": {
+            "type": "object",
+            "description": "Applied voice-performance contract for narration assets.",
+            "properties": {
+              "source_section_id": { "type": "string" },
+              "delivery_cues_applied": { "type": "boolean" },
+              "provider_text_used": { "type": "boolean" },
+              "provider_settings": {
+                "type": "object",
+                "additionalProperties": true
+              },
+              "sample_approved": { "type": "boolean" },
+              "sample_path": { "type": "string" },
+              "review_notes": { "type": "string" }
+            },
+            "additionalProperties": false
+          }
         },
         "additionalProperties": false
       }

schemas/artifacts/proposal_packet.schema.json

@@ -183,7 +183,19 @@
             "provider": { "type": "string" },
             "voice_id": { "type": "string" },
             "rationale": { "type": "string" },
-            "estimated_cost_usd": { "type": "number", "minimum": 0 }
+            "estimated_cost_usd": { "type": "number", "minimum": 0 },
+            "delivery_style": {
+              "type": "string",
+              "description": "Plain-language voice direction, e.g. warm expert, expressive narrator, crisp tutorial."
+            },
+            "pacing_policy": {
+              "type": "string",
+              "description": "How the voice should use pauses, speed, and emphasis across the piece."
+            },
+            "sample_approval_required": {
+              "type": "boolean",
+              "description": "True when a TTS sample must be approved before batch narration generation."
+            }
           },
           "additionalProperties": false
         },

schemas/artifacts/script.schema.json

@@ -9,6 +9,28 @@
     "version": { "type": "string", "const": "1.0" },
     "title": { "type": "string" },
     "total_duration_seconds": { "type": "number", "minimum": 1 },
+    "voice_performance": {
+      "type": "object",
+      "description": "Narration delivery contract used by TTS asset generation.",
+      "properties": {
+        "performance_intent": { "type": "string" },
+        "pacing_profile": {
+          "type": "string",
+          "enum": ["contemplative", "conversational", "energetic", "technical", "cinematic", "custom"]
+        },
+        "energy_curve": { "type": "string" },
+        "pause_policy": { "type": "string" },
+        "sample_section_id": {
+          "type": "string",
+          "description": "Most performance-sensitive section to use for TTS sample approval."
+        },
+        "provider_notes": {
+          "type": "object",
+          "additionalProperties": { "type": "string" }
+        }
+      },
+      "additionalProperties": false
+    },
     "sections": {
       "type": "array",
       "items": {
@@ -21,6 +43,29 @@
           "start_seconds": { "type": "number", "minimum": 0 },
           "end_seconds": { "type": "number", "minimum": 0 },
           "speaker_directions": { "type": "string" },
+          "delivery_cues": {
+            "type": "object",
+            "description": "Structured voice-performance cues that TTS generation must apply.",
+            "properties": {
+              "pace": {
+                "type": "string",
+                "enum": ["slow", "measured", "conversational", "brisk", "fast", "custom"]
+              },
+              "energy": { "type": "string" },
+              "emphasis_words": {
+                "type": "array",
+                "items": { "type": "string" }
+              },
+              "pause_before_seconds": { "type": "number", "minimum": 0 },
+              "pause_after_seconds": { "type": "number", "minimum": 0 },
+              "delivery_note": { "type": "string" },
+              "provider_text": {
+                "type": "string",
+                "description": "Provider-ready narration text, including purposeful punctuation or SSML break tags when supported."
+              }
+            },
+            "additionalProperties": false
+          },
           "enhancement_cues": {
             "type": "array",
             "items": {

skills/meta/voice-performance-director.md

@@ -0,0 +1,93 @@
+# Voice Performance Director
+
+Use this meta skill whenever a pipeline will generate narration with TTS.
+
+The goal is to make generated narration sound directed, not merely read. Do not
+leave expressiveness as "read naturally" in a prompt. Carry a concrete voice
+performance plan from script to asset generation, then verify it with a sample.
+
+## Required Contract
+
+Every narration-led script should include a top-level `voice_performance`
+object and section-level `delivery_cues` where the schema allows it.
+
+Top-level voice performance:
+
+```json
+{
+  "performance_intent": "Warm, decisive product narrator with human pauses.",
+  "pacing_profile": "conversational",
+  "energy_curve": "measured hook, warmer middle, more deliberate close",
+  "pause_policy": "Use short pauses after setup lines and longer pauses before reversals or important claims.",
+  "provider_notes": {
+    "openai": "Use instructions for emotional arc and emphasis.",
+    "google_tts": "Use SSML input with break tags when the selected voice supports it.",
+    "elevenlabs": "Use lower stability and moderate style for expressive narration."
+  }
+}
+```
+
+Section-level delivery cues:
+
+```json
+{
+  "pace": "measured",
+  "energy": "curious",
+  "emphasis_words": ["not", "process"],
+  "pause_before_seconds": 0.2,
+  "pause_after_seconds": 0.7,
+  "delivery_note": "Set up the contrast, then slow down on the final phrase.",
+  "provider_text": "This is not just another tool. <break time=\"0.6s\"/> It is a process."
+}
+```
+
+## Writing Rules
+
+- Write spoken language, not essay language. Prefer short sentences, light
+  contractions, and clear punctuation.
+- Use silence as structure. Add a pause before reversals, after surprising
+  claims, and before the final takeaway.
+- Keep pause tags purposeful. Too many breaks sound theatrical and slow.
+- Avoid generic directions such as "natural", "engaging", or "expressive"
+  unless they are paired with exact pace, emphasis, pause, or energy cues.
+- Prefer one delivery idea per section. If a section needs three emotional
+  turns, split it.
+
+## Provider Mapping
+
+- OpenAI TTS: use `model: "gpt-4o-mini-tts"` when sending `instructions`.
+  Put the emotional arc, pacing, emphasis, and role in `instructions`; keep the
+  input text clean but punctuated. Do not send `instructions` to `tts-1` or
+  `tts-1-hd`.
+- Google TTS: use `input_type: "ssml"` only when adding break tags or other
+  SSML. The tool maps this to Google `input.ssml` and wraps the utterance in
+  `<speak>...</speak>` when needed. Keep `speaking_rate` in Google's supported
+  `0.25..2.0` range and pitch in `-20..20`.
+- ElevenLabs: use lower `stability` for more variation, moderate `style` for
+  expressiveness, `speed` in the provider's `0.7..1.2` range, and keep
+  `similarity_boost` high enough to preserve the voice.
+- Offline/basic voices: rely on punctuation, shorter sentences, and explicit
+  segment splitting because provider-level emotion controls may be unavailable.
+
+## Sample Gate
+
+Before batch narration generation:
+
+1. Generate a sample from the most performance-sensitive section, not
+   automatically the first section.
+2. Verify voice, pace, pauses, emphasis, and emotional arc.
+3. If the sample is flat, adjust the `voice_performance` plan or provider
+   settings before generating the rest.
+4. Record the approved sample path and provider settings in the asset manifest.
+
+## Failure Conditions
+
+Treat these as quality failures:
+
+- A narration-led script has no `voice_performance` plan.
+- Section directions only say "read naturally" or "expressive" with no concrete
+  pause, emphasis, pace, or energy cue.
+- TTS provider, voice, speed, or model changes after sample approval without a
+  new sample.
+- Final narration is generated from raw script text while structured
+  `provider_text` or `delivery_cues` were present.

skills/pipelines/animation/asset-director.md

@@ -49,7 +49,7 @@ Prefer the lowest-variance useful path:
 
 Before batch-generating assets, produce one sample of each expensive type and show the user:
 
-1. **TTS sample** (if narration-led): Generate one section. Confirm voice and tone before batching.
+1. **TTS sample** (if narration-led): Generate `script.voice_performance.sample_section_id` when present; otherwise choose the section with the strongest emotional or pacing change. Confirm voice, pace, pauses, emphasis, and tone before batching.
 2. **Visual sample**: Generate one representative scene visual (diagram, illustration, or motion background). Confirm style and quality before batching the rest.
 
 If rejected, adjust parameters and retry (max 3 iterations). Do not batch until approved.
@@ -100,7 +100,12 @@ Create once:
 
 ### 3. Narration Is Optional, But The Plan Must Be Explicit
 
-If the project is narration-led, produce or source narration. If it is text-led or music-led, say so clearly in metadata.
+If the project is narration-led, produce or source narration. Read
+`skills/meta/voice-performance-director.md`, then apply `script.voice_performance`
+and each section's `delivery_cues` when building TTS requests. Use
+`provider_text` when present, map cues to provider controls, and record the
+applied settings on each narration asset. If it is text-led or music-led, say so
+clearly in metadata.
 
 ### 4. Use Metadata For Feasibility Truth
 
@@ -109,6 +114,7 @@ Recommended metadata keys:
 - `tool_path_map`
 - `reusable_assets`
 - `narration_assets`
+- `voice_performance`: sample approval path, provider settings, and whether delivery cues were applied
 - `scene_asset_index`
 - `blocked_assets`
 
@@ -117,7 +123,8 @@ Recommended metadata keys:
 - the asset path is explicit per scene,
 - reusable assets are actually reused,
 - missing capabilities are surfaced honestly,
-- every referenced file exists.
+- every referenced file exists,
+- narration-led assets apply the approved voice-performance settings.
 
 ### Mid-Production Fact Verification
 
@@ -135,6 +142,8 @@ the AI model's training data — it may be wrong or outdated.
 - Using high-variance generation when a deterministic asset would work better.
 - Rebuilding the same title or label system repeatedly.
 - Hiding failed asset paths instead of reporting them.
+- Treating TTS as raw text-to-audio. Narration-led animation needs pauses,
+  emphasis, and pace cues carried from the script into the generated audio.
 - Treating "consistency" as "same prompt every time." Good animation keeps a recognizable world while still letting each beat feel fresh.
 
 

skills/pipelines/animation/script-director.md

@@ -11,6 +11,7 @@ This stage turns the approved proposal into animation-ready beats. The script mu
 | Schema | `schemas/artifacts/script.schema.json` | Artifact validation |
 | Prior artifact | `proposal_packet` from Proposal Director | Selected concept, animation mode, target duration, reuse strategy |
 | Optional artifact | `research_brief` from Research Director | Data points, audience insights, accuracy constraints |
+| Meta skill | `skills/meta/voice-performance-director.md` | Structured TTS delivery cues for natural, expressive narration |
 | Tools | `transcriber` | Optional source transcript support |
 
 ## Process
@@ -41,6 +42,11 @@ Each section should express ONE clear visual idea:
 - **Comparison** — show two things side by side (split screen or sequential)
 - **Conclusion** — land the insight (hold + emphasis)
 
+If the piece is narration-led, also create `voice_performance` and section
+`delivery_cues` using `skills/meta/voice-performance-director.md`. Motion-heavy
+videos need vocal breathing room: mark pauses where the viewer must absorb an
+animation, and mark emphasis where a reveal, transform, or comparison lands.
+
 **Animation mode affects writing style:**
 
 | Mode | Writing Style |
@@ -82,6 +88,7 @@ Recommended metadata keys per section:
 - `narration_plan`: how narration relates to visual (describes / complements / silent)
 - `visual_priority`: what the viewer should focus on (the animation, the text, the data)
 - `hold_time_seconds`: minimum visual hold time after this section's content
+- `delivery_cues`: pace, energy, emphasis, pause timing, and provider-ready narration text for TTS
 - `data_source`: if this section uses a research data point, reference it
 
 ### 6. Research Integration
@@ -101,6 +108,7 @@ Before submitting the script, verify:
 - [ ] Every section supports ONE strong visual idea
 - [ ] On-screen text is concise (phrases, not paragraphs)
 - [ ] Timing is animation-friendly (holds budgeted)
+- [ ] Narration-led sections include concrete delivery cues and a voice-performance sample section
 - [ ] Word count is within ±10% of target duration
 - [ ] Animation mode is respected in writing style
 - [ ] Research data points are integrated (if research_brief available)

skills/pipelines/explainer/asset-director.md

@@ -70,7 +70,7 @@ Before generating anything:
 
 Before batch-generating assets, produce one sample of each expensive asset type and present them to the user for approval:
 
-1. **TTS sample**: Generate narration for the first script section only. Play it for the user. Confirm voice, pace, and tone are acceptable before generating the rest.
+1. **TTS sample**: Generate narration for `script.voice_performance.sample_section_id` when present; otherwise pick the section with the most demanding delivery. Play it for the user. Confirm voice, pace, pauses, emphasis, and tone are acceptable before generating the rest.
 2. **Image sample**: Generate one image for the most representative scene. Show it to the user. Confirm the style, quality, and prompt approach before batch-generating all images.
 3. **Music sample** (if using `music_gen`): Generate one short clip. Confirm mood and energy before committing.
 
@@ -85,13 +85,24 @@ This step typically costs $0.03–0.08 total and prevents $1–3 of wasted gener
 
 For each script section:
 1. Extract the narration text
-2. Apply speaker directions from the script (pace, emphasis, emotion)
-3. Apply the playbook's `audio.voice_style`
-4. Generate using `tts_selector` — it auto-routes to the best available TTS provider based on user preference and availability. Check the registry's `best_for` fields to understand each provider's strengths.
-5. Verify the audio file exists and duration matches expected timing (±15%)
+2. Read `script.voice_performance` and section `delivery_cues`
+3. Use `delivery_cues.provider_text` when present; otherwise transform the section text with purposeful punctuation and break tags only when the selected provider supports them
+4. Apply speaker directions from the script (pace, emphasis, emotion)
+5. Apply the playbook's `audio.voice_style`
+6. Map cues to provider parameters:
+   - OpenAI: `instructions` only with `model: "gpt-4o-mini-tts"`; use `response_format` for output format
+   - Google TTS: `input_type: "ssml"` when using `<break>` tags, plus `speaking_rate` in `0.25..2.0` and `pitch` in `-20..20`
+   - ElevenLabs: `stability`, `similarity_boost`, `style`, `speed`, and `use_speaker_boost`
+7. Generate using `tts_selector` — it auto-routes to the best available TTS provider based on user preference and availability. Check the registry's `best_for` fields to understand each provider's strengths.
+8. Record the applied `voice_performance` metadata on each narration asset
+9. Verify the audio file exists and duration matches expected timing (±15%)
 
 **Pronunciation guide**: If the script contains technical terms, jargon, or names with non-obvious pronunciation, include a pronunciation map in the TTS request.
 
+**Flat voice failure:** If the approved voice sounds monotone, robotic, rushed,
+or ignores intended pauses, do not batch the remaining sections. Revise the
+`voice_performance` plan or provider parameters and regenerate the sample.
+
 ### Step 4: Generate Visual Assets
 
 Process asset tasks grouped by tool for efficiency:
@@ -204,6 +215,8 @@ Assemble all generated assets into the manifest:
 
 **Quality check:**
 - [ ] Narration durations within ±15% of expected timing
+- [ ] Narration assets record `voice_performance.delivery_cues_applied`
+- [ ] Approved TTS sample uses the same provider, voice, and expressive settings as the batch
 - [ ] Images match the playbook's style (review consistency anchors)
 - [ ] Diagrams are legible and complete
 - [ ] Total cost within budget
@@ -242,6 +255,7 @@ the AI model's training data — it may be wrong or outdated.
 - **Generating before checking budget**: Always estimate total cost first. A 60-second video with 15 images can burn $3+ quickly.
 - **Inconsistent image style**: Each image_selector call is independent. Use consistent anchors, but adapt them per scene. If you paste the same style prefix into every prompt, the video will feel machine-made and repetitive.
 - **Ignoring narration timing**: If TTS produces 12s of audio for a 10s section, the edit phase will struggle. Check durations.
+- **Ignoring delivery cues**: Generating raw script text when `provider_text` or `delivery_cues` exist will flatten the read. Apply the voice-performance contract first.
 - **Missing pronunciation guide**: "PostgreSQL" or "Kubernetes" will be mispronounced without explicit guidance.
 - **One retry then give up**: If an image doesn't match, refine the prompt specifically — don't just retry the same prompt.
 - **AI-generating images with exact text (CTA, business names, contact info)**: AI image models frequently hallucinate wrong text — wrong business name, wrong phone number, misspelled words. **Never use AI image generation for scenes where text must be verbatim.** Use Remotion `text_card` type instead. This applies to: CTA screens, title cards with business names, contact info overlays, legal disclaimers. If a scene's `type` is `text_card` in the scene plan, do NOT generate an image for it — skip it and let the compose stage render it natively in Remotion.

skills/pipelines/explainer/script-director.md

@@ -14,6 +14,7 @@ The script is the backbone of the video. Every visual, every scene, every audio
 | Prior artifact | `proposal_packet` | Selected concept with title, hook, key_points, core_message, tone, narrative_structure, duration |
 | Prior artifact | `research_brief` (optional but high-value) | Data points, audience insights, expert quotes — ground the script in real facts |
 | Playbook | Active style playbook from `proposal_packet.selected_concept.suggested_playbook` | Voice style, pacing rules |
+| Meta skill | `skills/meta/voice-performance-director.md` | Structured TTS delivery cues for natural, expressive narration |
 | Layer 3 | TTS provider skills (check `agent_skills` on the selected TTS tool) | TTS capabilities for speaker directions |
 
 ## Process
@@ -77,6 +78,11 @@ Map each of the brief's `key_points` to a specific section in the BUILD phase.
 
 ### Step 4: Write the Script
 
+Before writing sections, create a top-level `voice_performance` plan using
+`skills/meta/voice-performance-director.md`. The plan must describe the vocal
+intent, pacing profile, energy curve, pause policy, and which section should be
+used for TTS sample approval. Do not leave this as a vague "natural voice" note.
+
 Write each section with these fields:
 
 ```json
@@ -87,6 +93,14 @@ Write each section with these fields:
   "start_seconds": 0,
   "end_seconds": 5,
   "speaker_directions": "Emphasize 'every single row' with measured pacing. Brief pause before the question.",
+  "delivery_cues": {
+    "pace": "measured",
+    "energy": "curious",
+    "emphasis_words": ["every", "single"],
+    "pause_after_seconds": 0.6,
+    "delivery_note": "Let the repetition feel intentional, then soften into the question.",
+    "provider_text": "Your database searches every single row. Every. Single. One. <break time=\"0.6s\"/> What if it didn't have to?"
+  },
   "enhancement_cues": [
     {
       "type": "animation",
@@ -117,7 +131,8 @@ Count your words. If you're 20%+ over budget, the TTS will either rush or exceed
 
 #### Speaker Directions
 
-Write directions that TTS can actually implement. Reference ElevenLabs capabilities:
+Write directions that TTS can actually implement. Prefer structured
+`delivery_cues` over prose-only `speaker_directions`:
 
 | Direction | TTS Implementation |
 |-----------|-------------------|
@@ -129,6 +144,12 @@ Write directions that TTS can actually implement. Reference ElevenLabs capabilit
 
 Avoid directions TTS can't do: "smile while speaking", "gesture toward screen", "look at camera."
 
+**Expressive narration rule:** every narration-led section must include at
+least two concrete cues among `pace`, `energy`, `emphasis_words`,
+`pause_before_seconds`, `pause_after_seconds`, `delivery_note`, or
+`provider_text`. Use `provider_text` when punctuation or SSML break tags are
+needed to make the read sound human.
+
 #### Enhancement Cues
 
 Every section should have at least one enhancement cue. These tell the Scene Planner and Asset Generator what visuals to create.
@@ -162,6 +183,7 @@ Read the active style playbook and verify:
 |----------------|---------------|
 | `identity.pace` | Match word density. `contemplative` = fewer words, longer pauses |
 | `audio.voice_style` | Shape tone of speaker directions |
+| `voice_performance` | Confirm pacing, pauses, and energy curve are explicit enough for TTS |
 | `motion.pacing_rules` | E.g., "hold establishing shots for 2s minimum" affects section timing |
 | `identity.mood` | Word choice: `warm` uses casual language; `professional` uses precise language |
 
@@ -175,6 +197,7 @@ Score your script (1-5):
 | **Word count accuracy** | Within ±10% of target for the duration? |
 | **Narrative flow** | Does each section build on the last? "Therefore/but" not "and then"? |
 | **Enhancement density** | At least one cue every 8-10 seconds? |
+| **Voice performance** | Are pauses, emphasis, pace, and sample section explicit? |
 | **Jargon management** | Technical terms explained or have pronunciation guides? |
 | **Climax payoff** | Does the aha moment deliver on the hook's promise? |
 | **CTA relevance** | Is the call to action specific and actionable? |

tests/tools/test_hyperframes_compose.py

@@ -1062,6 +1062,69 @@ def test_proposal_packet_schema_accepts_render_runtime():
     assert props["render_runtime"]["enum"] == ["remotion", "hyperframes", "ffmpeg"]
 
 
+def test_schemas_accept_voice_performance_contract():
+    root = Path(__file__).resolve().parent.parent.parent
+
+    script_schema = json.loads(
+        (root / "schemas" / "artifacts" / "script.schema.json").read_text(
+            encoding="utf-8"
+        )
+    )
+    assert "voice_performance" in script_schema["properties"]
+    section_props = script_schema["properties"]["sections"]["items"]["properties"]
+    assert "delivery_cues" in section_props
+    assert "provider_text" in section_props["delivery_cues"]["properties"]
+
+    proposal_schema = json.loads(
+        (root / "schemas" / "artifacts" / "proposal_packet.schema.json").read_text(
+            encoding="utf-8"
+        )
+    )
+    voice_selection = proposal_schema["properties"]["production_plan"]["properties"][
+        "voice_selection"
+    ]["properties"]
+    assert "delivery_style" in voice_selection
+    assert "pacing_policy" in voice_selection
+    assert "sample_approval_required" in voice_selection
+
+    asset_schema = json.loads(
+        (root / "schemas" / "artifacts" / "asset_manifest.schema.json").read_text(
+            encoding="utf-8"
+        )
+    )
+    asset_props = asset_schema["properties"]["assets"]["items"]["properties"]
+    assert "voice_performance" in asset_props
+    assert "provider_settings" in asset_props["voice_performance"]["properties"]
+
+
+def test_tts_provider_contracts_match_supported_fields():
+    from tools.audio.elevenlabs_tts import ElevenLabsTTS
+    from tools.audio.google_tts import GoogleTTS
+    from tools.audio.openai_tts import OpenAITTS
+
+    google_props = GoogleTTS.input_schema["properties"]
+    assert google_props["input_type"]["enum"] == ["text", "ssml"]
+    assert google_props["speaking_rate"]["maximum"] == 2.0
+    assert google_props["pitch"]["minimum"] == -20.0
+    assert google_props["pitch"]["maximum"] == 20.0
+
+    openai_props = OpenAITTS.input_schema["properties"]
+    assert "response_format" in openai_props
+    assert {"mp3", "opus", "aac", "flac", "wav", "pcm"}.issubset(
+        set(openai_props["response_format"]["enum"])
+    )
+    assert OpenAITTS._supports_instructions("gpt-4o-mini-tts")
+    assert not OpenAITTS._supports_instructions("tts-1")
+    assert not OpenAITTS._supports_instructions("tts-1-hd")
+
+    eleven_props = ElevenLabsTTS.input_schema["properties"]
+    assert {"stability", "similarity_boost", "style", "speed", "use_speaker_boost"}.issubset(
+        set(eleven_props)
+    )
+    assert eleven_props["speed"]["minimum"] == 0.7
+    assert eleven_props["speed"]["maximum"] == 1.2
+
+
 def test_edit_decisions_schema_accepts_render_runtime():
     schema_path = (
         Path(__file__).resolve().parent.parent.parent

tools/audio/elevenlabs_tts.py

@@ -96,6 +96,16 @@ class ElevenLabsTTS(BaseTool):
                 "minimum": 0,
                 "maximum": 1,
             },
+            "speed": {
+                "type": "number",
+                "default": 1.0,
+                "minimum": 0.7,
+                "maximum": 1.2,
+            },
+            "use_speaker_boost": {
+                "type": "boolean",
+                "default": True,
+            },
             "output_path": {"type": "string"},
             "output_format": {
                 "type": "string",
@@ -109,7 +119,16 @@ class ElevenLabsTTS(BaseTool):
         cpu_cores=1, ram_mb=256, vram_mb=0, disk_mb=50, network_required=True
     )
     retry_policy = RetryPolicy(max_retries=2, retryable_errors=["rate_limit", "timeout"])
-    idempotency_key_fields = ["text", "voice_id", "model_id"]
+    idempotency_key_fields = [
+        "text",
+        "voice_id",
+        "model_id",
+        "stability",
+        "similarity_boost",
+        "style",
+        "speed",
+        "use_speaker_boost",
+    ]
     side_effects = ["writes audio file to output_path", "calls ElevenLabs API"]
     user_visible_verification = ["Listen to generated audio for natural speech quality"]
 
@@ -145,6 +164,13 @@ class ElevenLabsTTS(BaseTool):
         voice_id = inputs.get("voice_id", self.DEFAULT_VOICE_ID)
         model_id = inputs.get("model_id", "eleven_multilingual_v2")
         output_format = inputs.get("output_format", "mp3_44100_128")
+        voice_settings = {
+            "stability": inputs.get("stability", 0.5),
+            "similarity_boost": inputs.get("similarity_boost", 0.75),
+            "style": inputs.get("style", 0.0),
+            "speed": inputs.get("speed", 1.0),
+            "use_speaker_boost": inputs.get("use_speaker_boost", True),
+        }
 
         response = requests.post(
             f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}",
@@ -156,11 +182,7 @@ class ElevenLabsTTS(BaseTool):
             json={
                 "text": text,
                 "model_id": model_id,
-                "voice_settings": {
-                    "stability": inputs.get("stability", 0.5),
-                    "similarity_boost": inputs.get("similarity_boost", 0.75),
-                    "style": inputs.get("style", 0.0),
-                },
+                "voice_settings": voice_settings,
             },
             params={"output_format": output_format},
             timeout=120,
@@ -178,6 +200,7 @@ class ElevenLabsTTS(BaseTool):
                 "provider": self.provider,
                 "model": model_id,
                 "voice_id": voice_id,
+                "voice_settings": voice_settings,
                 "text_length": len(text),
                 "output": str(output_path),
                 "format": output_format,

tools/audio/google_tts.py

@@ -78,6 +78,12 @@ class GoogleTTS(BaseTool):
         "required": ["text"],
         "properties": {
             "text": {"type": "string", "description": "Text to convert to speech"},
+            "input_type": {
+                "type": "string",
+                "default": "text",
+                "enum": ["text", "ssml"],
+                "description": "Set to 'ssml' when text contains SSML tags such as <speak> or <break>.",
+            },
             "voice": {
                 "type": "string",
                 "default": "en-US-Chirp3-HD-Orus",
@@ -92,7 +98,7 @@ class GoogleTTS(BaseTool):
                 "type": "number",
                 "default": 1.0,
                 "minimum": 0.25,
-                "maximum": 4.0,
+                "maximum": 2.0,
                 "description": "Speaking speed. 1.0 = normal, 0.5 = half speed, 2.0 = double speed",
             },
             "pitch": {
@@ -116,7 +122,7 @@ class GoogleTTS(BaseTool):
         cpu_cores=1, ram_mb=256, vram_mb=0, disk_mb=50, network_required=True
     )
     retry_policy = RetryPolicy(max_retries=2, retryable_errors=["rate_limit", "timeout"])
-    idempotency_key_fields = ["text", "voice", "language_code", "speaking_rate", "pitch"]
+    idempotency_key_fields = ["text", "input_type", "voice", "language_code", "speaking_rate", "pitch"]
     side_effects = ["writes audio file to output_path", "calls Google Cloud TTS API"]
     user_visible_verification = ["Listen to generated audio for natural speech quality"]
 
@@ -200,14 +206,33 @@ class GoogleTTS(BaseTool):
         import requests
 
         text = inputs["text"]
+        input_type = inputs.get("input_type", "text")
         voice_name = inputs.get("voice", "en-US-Chirp3-HD-Orus")
         language_code = inputs.get("language_code", "en-US")
         speaking_rate = inputs.get("speaking_rate", 1.0)
         pitch = inputs.get("pitch", 0.0)
         audio_encoding = inputs.get("audio_encoding", "MP3")
 
+        if not 0.25 <= speaking_rate <= 2.0:
+            return ToolResult(
+                success=False,
+                error="Google TTS speaking_rate must be between 0.25 and 2.0.",
+            )
+        if not -20.0 <= pitch <= 20.0:
+            return ToolResult(
+                success=False,
+                error="Google TTS pitch must be between -20.0 and 20.0 semitones.",
+            )
+
+        if input_type == "ssml":
+            stripped = text.strip()
+            ssml = stripped if stripped.startswith("<speak") else f"<speak>{stripped}</speak>"
+            synthesis_input = {"ssml": ssml}
+        else:
+            synthesis_input = {"text": text}
+
         payload = {
-            "input": {"text": text},
+            "input": synthesis_input,
             "voice": {
                 "languageCode": language_code,
                 "name": voice_name,
@@ -253,6 +278,7 @@ class GoogleTTS(BaseTool):
                 "voice": voice_name,
                 "language_code": language_code,
                 "text_length": len(text),
+                "input_type": input_type,
                 "output": str(output_path),
                 "format": audio_encoding,
                 "speaking_rate": speaking_rate,

tools/audio/openai_tts.py

@@ -80,10 +80,24 @@ class OpenAITTS(BaseTool):
                 "type": "string",
                 "default": "mp3",
                 "enum": ["mp3", "wav", "pcm"],
+                "description": "Backward-compatible alias for response_format.",
+            },
+            "response_format": {
+                "type": "string",
+                "default": "mp3",
+                "enum": ["mp3", "opus", "aac", "flac", "wav", "pcm"],
+                "description": "OpenAI speech response_format.",
             },
             "instructions": {
                 "type": "string",
-                "description": "Optional delivery instructions for the voice",
+                "description": "Optional delivery instructions. Supported by gpt-4o-mini-tts.",
+            },
+            "speed": {
+                "type": "number",
+                "default": 1.0,
+                "minimum": 0.25,
+                "maximum": 4.0,
+                "description": "OpenAI speech speed multiplier.",
             },
             "output_path": {"type": "string"},
         },
@@ -93,7 +107,7 @@ class OpenAITTS(BaseTool):
         cpu_cores=1, ram_mb=256, vram_mb=0, disk_mb=50, network_required=True
     )
     retry_policy = RetryPolicy(max_retries=2, retryable_errors=["rate_limit", "timeout"])
-    idempotency_key_fields = ["text", "voice", "model", "format"]
+    idempotency_key_fields = ["text", "voice", "model", "format", "response_format", "instructions", "speed"]
     side_effects = ["writes audio file to output_path", "calls OpenAI API"]
     user_visible_verification = ["Listen to generated audio for intelligibility and tone"]
 
@@ -105,6 +119,10 @@ class OpenAITTS(BaseTool):
     def estimate_cost(self, inputs: dict[str, Any]) -> float:
         return round(len(inputs.get("text", "")) * 0.000015, 4)
 
+    @staticmethod
+    def _supports_instructions(model: str) -> bool:
+        return model.startswith("gpt-4o-mini-tts")
+
     def execute(self, inputs: dict[str, Any]) -> ToolResult:
         if not os.environ.get("OPENAI_API_KEY"):
             return ToolResult(success=False, error="No OpenAI API key. " + self.install_instructions)
@@ -124,11 +142,20 @@ class OpenAITTS(BaseTool):
 
         from tools.analysis.audio_probe import probe_duration
 
-        client = OpenAI()
         text = inputs["text"]
         model = inputs.get("model", "gpt-4o-mini-tts")
         voice = inputs.get("voice", "alloy")
-        fmt = inputs.get("format", "mp3")
+        fmt = inputs.get("response_format") or inputs.get("format", "mp3")
+        if inputs.get("instructions") and not self._supports_instructions(model):
+            return ToolResult(
+                success=False,
+                error=(
+                    "OpenAI TTS instructions are only supported by "
+                    "gpt-4o-mini-tts. Use that model or omit instructions."
+                ),
+            )
+
+        client = OpenAI()
         output_path = Path(inputs.get("output_path", f"openai_tts.{fmt}"))
         output_path.parent.mkdir(parents=True, exist_ok=True)
 
@@ -155,6 +182,9 @@ class OpenAITTS(BaseTool):
                 "model": model,
                 "voice": voice,
                 "format": fmt,
+                "response_format": fmt,
+                "instructions": inputs.get("instructions"),
+                "speed": inputs.get("speed", 1.0),
                 "text_length": len(text),
                 "audio_duration_seconds": round(audio_duration, 2) if audio_duration else None,
                 "output": str(output_path),

tools/audio/tts_selector.py

@@ -61,6 +61,43 @@ class TTSSelector(BaseTool):
                 "type": "number", "minimum": 0, "maximum": 1,
                 "description": "Style exaggeration (ElevenLabs). Higher = more expressive.",
             },
+            "instructions": {
+                "type": "string",
+                "description": "Provider-level delivery instructions for expressive narration when supported.",
+            },
+            "speaking_rate": {
+                "type": "number",
+                "minimum": 0.25,
+                "maximum": 2.0,
+                "description": "Google-style speakingRate control. Use speed for OpenAI/ElevenLabs-style controls.",
+            },
+            "speed": {
+                "type": "number",
+                "minimum": 0.25,
+                "maximum": 4.0,
+                "description": "Alias for speaking speed used by some providers.",
+            },
+            "pitch": {
+                "type": "number",
+                "minimum": -50,
+                "maximum": 50,
+                "description": "Provider-specific pitch control. Google TTS accepts -20..20; HeyGen-style providers may accept wider ranges.",
+            },
+            "input_type": {
+                "type": "string",
+                "enum": ["text", "ssml"],
+                "default": "text",
+                "description": "Use 'ssml' only when the selected provider supports tags such as <break>.",
+            },
+            "voice_performance": {
+                "type": "object",
+                "description": "Structured voice-performance plan or section delivery cues from the script artifact.",
+            },
+            "sample_mode": {
+                "type": "boolean",
+                "default": False,
+                "description": "True when generating an approval sample before batch narration.",
+            },
             "output_format": {
                 "type": "string",
                 "description": "Audio output format (e.g. mp3_44100_128). Passed through to provider.",