Merge pull request #354 from amartya-dev/feat/azure-speech-to-text

feat(stt): add Azure AI Speech as an optional cloud speech-to-text provider
2026-08-05 15:20:40 +08:00 · 2026-07-12 10:48:36 -07:00
parent 94f382a1b4 a2a0d8c8af
commit f8d94632ea
8 changed files with 900 additions and 3 deletions
--- a/.agents/skills/azure-speech-to-text/SKILL.md
+++ b/.agents/skills/azure-speech-to-text/SKILL.md
@@ -0,0 +1,115 @@
+---
+name: azure-speech-to-text
+description: Transcribe audio to text using Azure AI Speech (Fast Transcription REST API). Use when converting audio/video to text, generating subtitles, or processing spoken content in OpenMontage. Optional cloud STT provider — preferred when AZURE_SPEECH_KEY is configured; the local faster-whisper `transcriber` is the default offline path.
+license: MIT
+compatibility: Requires internet access and an Azure AI Speech resource (AZURE_SPEECH_KEY + AZURE_SPEECH_REGION).
+metadata: {"openclaw": {"requires": {"env": ["AZURE_SPEECH_KEY", "AZURE_SPEECH_REGION"]}, "primaryEnv": "AZURE_SPEECH_KEY"}}
+---
+
+# Azure AI Speech — Speech-to-Text
+
+Transcribe audio to text with **Azure Fast Transcription** — synchronous,
+word-level timestamps, speaker diarization, and multi-language identification.
+In OpenMontage this is exposed through the `azure_stt` tool (`capability=analysis`,
+`provider=azure`). It is an **optional cloud STT provider** — when
+`AZURE_SPEECH_KEY` is configured, prefer it for cloud transcription. The local
+`transcriber` tool (faster-whisper) remains the **default offline path** and the
+fallback when Azure is unavailable.
+
+> Docs: [Fast Transcription](https://learn.microsoft.com/azure/ai-services/speech-service/fast-transcription-create) · [Speech service overview](https://learn.microsoft.com/azure/ai-services/speech-service/spx-overview)
+
+## Why Fast Transcription (not Batch)
+
+Azure exposes three STT surfaces. OpenMontage uses **Fast Transcription** because
+the pipeline transcribes **local audio files**:
+
+| Surface | Input | Latency | Needs |
+|---------|-------|---------|-------|
+| **Fast Transcription** (used here) | local file, multipart POST | synchronous, sub-real-time | key + region |
+| Batch Transcription | audio at a URL (Blob + SAS) | async job + polling | Blob storage plumbing |
+| Speech SDK (`spx`) | mic / stream / file | streaming | native `azure-cognitiveservices-speech` package |
+
+Fast Transcription needs no Blob storage, no SAS URLs, and no native SDK — just
+`requests` and the two env vars.
+
+## Setup
+
+Create a **Speech** resource in the [Azure portal](https://portal.azure.com);
+copy the key and region from its **Keys and Endpoint** page.
+
+```bash
+export AZURE_SPEECH_KEY=your_speech_resource_key
+export AZURE_SPEECH_REGION=eastus          # your resource's region
+# export AZURE_SPEECH_ENDPOINT=https://...  # optional: overrides region
+```
+
+`azure_stt` reports `AVAILABLE` once `AZURE_SPEECH_KEY` plus either
+`AZURE_SPEECH_REGION` or `AZURE_SPEECH_ENDPOINT` are set.
+
+## Using it in a pipeline
+
+Prefer `azure_stt` over `transcriber` unless the run must be offline. Its output
+matches the `transcriber` schema exactly, so it is a drop-in for `subtitle_gen`
+and any stage that consumes a transcript.
+
+```python
+from tools.tool_registry import registry
+registry.discover()
+stt = registry._tools["azure_stt"]
+
+result = stt.execute({
+    "input_path": "projects/my-video/assets/audio/narration.mp3",
+    # "language": "en",          # ISO 639-1 or BCP-47 ("en-US"); omit for auto-ID
+    # "diarize": True,           # speaker labels, no HuggingFace token needed
+    # "max_speakers": 4,
+    "output_dir": "projects/my-video/artifacts",
+})
+if result.success:
+    segs = result.data["segments"]          # [{id,start,end,text,words:[...]}]
+    words = result.data["word_timestamps"]  # flat [{word,start,end,probability}]
+```
+
+If `azure_stt` is unavailable (no key) or errors, fall back to `transcriber`
+(local whisper) — its `execute` signature and output are identical.
+
+## Parameters that matter
+
+- **`language`** — pass an ISO code (`"en"`) or a full locale (`"en-US"`). Pin it
+  when you know the language; it is faster and more accurate than auto-ID.
+- **`candidate_locales`** — when `language` is omitted, Azure runs language
+  identification across this shortlist. Narrow it to the languages you actually
+  expect; a huge list slows detection and invites misclassification.
+- **`diarize` / `max_speakers`** — enable for multi-speaker audio (interviews,
+  podcasts). Set `max_speakers` to the real upper bound.
+- **`profanity_filter`** — `None` | `Masked` (default) | `Removed` | `Tags`.
+
+## Response shape (mapped to the transcriber schema)
+
+The raw Azure response (`phrases[]` with `offsetMilliseconds` / `words[]`) is
+converted to seconds and the OpenMontage transcript schema:
+
+```json
+{
+  "segments": [
+    {"id": 0, "start": 0.0, "end": 2.4, "text": "Hello world",
+     "speaker": 1,
+     "words": [{"word": "Hello", "start": 0.0, "end": 0.5, "probability": 0.98}]}
+  ],
+  "word_timestamps": [{"word": "Hello", "start": 0.0, "end": 0.5, "probability": 0.98}],
+  "language": "en-US",
+  "duration_seconds": 2.4,
+  "provider": "azure"
+}
+```
+
+Note: Fast Transcription has no *per-word* confidence, so each word carries the
+**phrase** confidence in `probability`.
+
+## Limits & tips
+
+- Single file up to ~2 hours / a few hundred MB per request. For longer or bulk
+  jobs, use Azure Batch Transcription instead.
+- Send clean audio (16 kHz+ mono is plenty). Transcode video to audio first if
+  you only need speech — smaller upload, same result.
+- Verify timing: word timestamps drive subtitle cues in `subtitle_gen`. Spot-check
+  the first and last cues against the source audio.
--- a/.claude/skills/azure-speech-to-text/SKILL.md
+++ b/.claude/skills/azure-speech-to-text/SKILL.md
@@ -0,0 +1,115 @@
+---
+name: azure-speech-to-text
+description: Transcribe audio to text using Azure AI Speech (Fast Transcription REST API). Use when converting audio/video to text, generating subtitles, or processing spoken content in OpenMontage. Optional cloud STT provider — preferred when AZURE_SPEECH_KEY is configured; the local faster-whisper `transcriber` is the default offline path.
+license: MIT
+compatibility: Requires internet access and an Azure AI Speech resource (AZURE_SPEECH_KEY + AZURE_SPEECH_REGION).
+metadata: {"openclaw": {"requires": {"env": ["AZURE_SPEECH_KEY", "AZURE_SPEECH_REGION"]}, "primaryEnv": "AZURE_SPEECH_KEY"}}
+---
+
+# Azure AI Speech — Speech-to-Text
+
+Transcribe audio to text with **Azure Fast Transcription** — synchronous,
+word-level timestamps, speaker diarization, and multi-language identification.
+In OpenMontage this is exposed through the `azure_stt` tool (`capability=analysis`,
+`provider=azure`). It is an **optional cloud STT provider** — when
+`AZURE_SPEECH_KEY` is configured, prefer it for cloud transcription. The local
+`transcriber` tool (faster-whisper) remains the **default offline path** and the
+fallback when Azure is unavailable.
+
+> Docs: [Fast Transcription](https://learn.microsoft.com/azure/ai-services/speech-service/fast-transcription-create) · [Speech service overview](https://learn.microsoft.com/azure/ai-services/speech-service/spx-overview)
+
+## Why Fast Transcription (not Batch)
+
+Azure exposes three STT surfaces. OpenMontage uses **Fast Transcription** because
+the pipeline transcribes **local audio files**:
+
+| Surface | Input | Latency | Needs |
+|---------|-------|---------|-------|
+| **Fast Transcription** (used here) | local file, multipart POST | synchronous, sub-real-time | key + region |
+| Batch Transcription | audio at a URL (Blob + SAS) | async job + polling | Blob storage plumbing |
+| Speech SDK (`spx`) | mic / stream / file | streaming | native `azure-cognitiveservices-speech` package |
+
+Fast Transcription needs no Blob storage, no SAS URLs, and no native SDK — just
+`requests` and the two env vars.
+
+## Setup
+
+Create a **Speech** resource in the [Azure portal](https://portal.azure.com);
+copy the key and region from its **Keys and Endpoint** page.
+
+```bash
+export AZURE_SPEECH_KEY=your_speech_resource_key
+export AZURE_SPEECH_REGION=eastus          # your resource's region
+# export AZURE_SPEECH_ENDPOINT=https://...  # optional: overrides region
+```
+
+`azure_stt` reports `AVAILABLE` once `AZURE_SPEECH_KEY` plus either
+`AZURE_SPEECH_REGION` or `AZURE_SPEECH_ENDPOINT` are set.
+
+## Using it in a pipeline
+
+Prefer `azure_stt` over `transcriber` unless the run must be offline. Its output
+matches the `transcriber` schema exactly, so it is a drop-in for `subtitle_gen`
+and any stage that consumes a transcript.
+
+```python
+from tools.tool_registry import registry
+registry.discover()
+stt = registry._tools["azure_stt"]
+
+result = stt.execute({
+    "input_path": "projects/my-video/assets/audio/narration.mp3",
+    # "language": "en",          # ISO 639-1 or BCP-47 ("en-US"); omit for auto-ID
+    # "diarize": True,           # speaker labels, no HuggingFace token needed
+    # "max_speakers": 4,
+    "output_dir": "projects/my-video/artifacts",
+})
+if result.success:
+    segs = result.data["segments"]          # [{id,start,end,text,words:[...]}]
+    words = result.data["word_timestamps"]  # flat [{word,start,end,probability}]
+```
+
+If `azure_stt` is unavailable (no key) or errors, fall back to `transcriber`
+(local whisper) — its `execute` signature and output are identical.
+
+## Parameters that matter
+
+- **`language`** — pass an ISO code (`"en"`) or a full locale (`"en-US"`). Pin it
+  when you know the language; it is faster and more accurate than auto-ID.
+- **`candidate_locales`** — when `language` is omitted, Azure runs language
+  identification across this shortlist. Narrow it to the languages you actually
+  expect; a huge list slows detection and invites misclassification.
+- **`diarize` / `max_speakers`** — enable for multi-speaker audio (interviews,
+  podcasts). Set `max_speakers` to the real upper bound.
+- **`profanity_filter`** — `None` | `Masked` (default) | `Removed` | `Tags`.
+
+## Response shape (mapped to the transcriber schema)
+
+The raw Azure response (`phrases[]` with `offsetMilliseconds` / `words[]`) is
+converted to seconds and the OpenMontage transcript schema:
+
+```json
+{
+  "segments": [
+    {"id": 0, "start": 0.0, "end": 2.4, "text": "Hello world",
+     "speaker": 1,
+     "words": [{"word": "Hello", "start": 0.0, "end": 0.5, "probability": 0.98}]}
+  ],
+  "word_timestamps": [{"word": "Hello", "start": 0.0, "end": 0.5, "probability": 0.98}],
+  "language": "en-US",
+  "duration_seconds": 2.4,
+  "provider": "azure"
+}
+```
+
+Note: Fast Transcription has no *per-word* confidence, so each word carries the
+**phrase** confidence in `probability`.
+
+## Limits & tips
+
+- Single file up to ~2 hours / a few hundred MB per request. For longer or bulk
+  jobs, use Azure Batch Transcription instead.
+- Send clean audio (16 kHz+ mono is plenty). Transcode video to audio first if
+  you only need speech — smaller upload, same result.
+- Verify timing: word timestamps drive subtitle cues in `subtitle_gen`. Spot-check
+  the first and last cues against the source audio.
--- a/.env.example
+++ b/.env.example
@@ -61,6 +61,12 @@ UNSPLASH_ACCESS_KEY=         # Unsplash stock images (free developer key)

 # --- Analysis ---
 HF_TOKEN=                    # HuggingFace token — enables speaker diarization in transcriber
+# Speech-to-text: optional Azure AI Speech (Fast Transcription). When set, the
+# agent prefers azure_stt for cloud STT; the local faster-whisper transcriber
+# remains the default offline path.
+AZURE_SPEECH_KEY=            # Azure AI Speech resource key ('Keys and Endpoint' page)
+AZURE_SPEECH_REGION=         # Speech resource region, e.g. eastus
+# AZURE_SPEECH_ENDPOINT=     # Optional: full custom endpoint URL (overrides region)

 # --- Avatar (local installs) ---
 # WAV2LIP_PATH=              # Path to cloned Wav2Lip repo (for lip sync)
--- a/AGENT_GUIDE.md
+++ b/AGENT_GUIDE.md
@@ -678,7 +678,8 @@ The `.agents/skills/` directory is large. When you're not coming in through a to
 | **Image generation** | `bfl-api`, `flux-best-practices` |
 | **Video generation** | `seedance-2-0` (preferred premium default — cinematic, trailer, multi-shot, synced audio, lip-sync), `gemini-omni` (conversational video editing, reference tags, timecoded beats), `ai-video-gen`, `ltx2` |
 | **Audio** | `elevenlabs`, `music`, `sound-effects`, `acestep`, `text-to-speech`, `setup-api-key` |
-| **Avatar / lip-sync** | `avatar-video`, `heygen`, `create-video`, `faceswap`, `video-translate`, `speech-to-text`, `agents` |
+| **Speech-to-text** | `speech-to-text` (whisper `transcriber` — default, offline), `azure-speech-to-text` (optional cloud STT — tool `azure_stt`, preferred when `AZURE_SPEECH_KEY` is set) |
+| **Avatar / lip-sync** | `avatar-video`, `heygen`, `create-video`, `faceswap`, `video-translate`, `agents` |
 | **Capture** | `playwright-recording` (browser flows), `ffmpeg` (post) |
 | **Visualization** | `beautiful-mermaid`, `d3-viz`, `manim-composer`, `manimce-best-practices`, `manimgl-best-practices` |
 | **Media editing** | `video-edit`, `video-download`, `video-understand`, `video-toolkit`, `visual-style` |
--- a/docs/PROVIDERS.md
+++ b/docs/PROVIDERS.md
@@ -44,6 +44,10 @@ DOUBAO_SPEECH_API_KEY=       # Volcengine Doubao Speech TTS (strong Mandarin nar
 DOUBAO_SPEECH_VOICE_TYPE=    # Default Doubao speaker/voice type
 DASHSCOPE_API_KEY=           # Alibaba DashScope (Qwen image gen, TTS, ASR with word timestamps)

+# SPEECH-TO-TEXT (optional cloud transcription; local whisper is the default)
+AZURE_SPEECH_KEY=            # Azure AI Speech — Fast Transcription (word-level timestamps)
+AZURE_SPEECH_REGION=         # Speech resource region, e.g. eastus
+
 # MULTI-MODEL GATEWAY (one key, 6+ tools)
 FAL_KEY=                     # FLUX, Recraft, Kling, Veo, MiniMax video

@@ -292,6 +296,54 @@ Doubao Speech 2.0 is billed by character package or usage in Volcengine. OpenMon

 ---

+### Azure AI Speech — Speech-to-Text
+
+> **Cloud transcription.** Azure AI Speech Fast Transcription turns local audio into text with word-level timestamps, speaker diarization, and multi-language identification — no GPU required. Optional: the local faster-whisper `transcriber` remains the default offline STT path. When `AZURE_SPEECH_KEY` is set, the agent prefers `azure_stt` for cloud transcription.
+
+**Tools unlocked:** `azure_stt`
+**Env vars:** `AZURE_SPEECH_KEY`, `AZURE_SPEECH_REGION` (or `AZURE_SPEECH_ENDPOINT`)
+
+#### Setup
+
+1. In the [Azure portal](https://portal.azure.com), create a **Speech** resource (Azure AI services → Speech service).
+2. Open the resource's **Keys and Endpoint** page.
+3. Copy **KEY 1** and the **Location/Region** (e.g. `eastus`).
+4. Add to `.env`:
+   ```bash
+   AZURE_SPEECH_KEY=your-speech-resource-key
+   AZURE_SPEECH_REGION=eastus
+   # AZURE_SPEECH_ENDPOINT=https://<custom>...  # optional, overrides region
+   ```
+
+#### API Notes
+
+OpenMontage uses the **Fast Transcription** REST endpoint, which accepts a local
+audio file directly (multipart upload) and returns a synchronous result — no
+Azure Blob storage, SAS URLs, or async job polling:
+
+```text
+POST https://{region}.api.cognitive.microsoft.com/speechtotext/transcriptions:transcribe?api-version=2024-11-15
+Ocp-Apim-Subscription-Key: ${AZURE_SPEECH_KEY}
+```
+
+For files longer than ~2 hours or bulk jobs, use Azure Batch Transcription instead (not wired into OpenMontage).
+
+#### What It Is Best For
+
+- Cloud transcription with word-level timestamps and no local GPU
+- Multi-language auto-detection across a candidate locale set
+- Speaker diarization without a HuggingFace token
+- Subtitle timing metadata that flows straight into `subtitle_gen`
+
+#### Pricing
+
+Azure AI Speech Standard (S0) bills speech-to-text by audio-hour (roughly
+$1.00/audio-hour at time of writing; a free F0 tier includes a limited monthly
+allowance). OpenMontage estimates cost from the transcribed audio duration. See
+[Azure AI Speech pricing](https://azure.microsoft.com/pricing/details/cognitive-services/speech-services/) for current rates.
+
+---
+
 ### Google — TTS + Imagen + Music + Video (Shared Key)

 > **One key, five tools.** Google Cloud TTS has 700+ voices in 50+ languages — the strongest localization option. Imagen 4 generates high-quality images. Google Lyria generates high-quality background music. Gemini Omni Flash supports conversational video editing, and direct Veo generation covers premium short video clips.
--- a/skills/INDEX.md
+++ b/skills/INDEX.md
@@ -81,7 +81,8 @@ Key capability families to look for in the output:
 | FFmpeg | `core/ffmpeg.md` | Video encoding, filtering, composition | `ffmpeg`, `video-toolkit` |
 | Remotion | `core/remotion.md` | React-based composition, Phase 3+ | `remotion-best-practices`, `remotion` |
 | HyperFrames | `core/hyperframes.md` | HTML/CSS/GSAP composition runtime — kinetic typography, music-to-video, product promos, website capture. Vendored at v0.7.17 (2026-06-27). | `hyperframes` (router) → `hyperframes-core` (contract), `hyperframes-creative` (palette/type/narration), `hyperframes-media` (TTS/BGM/SFX/captions), `hyperframes-animation` (all motion), `hyperframes-cli`, `hyperframes-registry`, `media-use`, `motion-graphics`, `music-to-video` (beats-driven), `website-to-video`, `remotion-to-hyperframes` (migration), `gsap-core`, `gsap-timeline` |
-| WhisperX | `core/whisperx.md` | Transcription with word-level timestamps | `speech-to-text` |
+| WhisperX | `core/whisperx.md` | Transcription with word-level timestamps — default STT (offline, free) | `speech-to-text` |
+| Azure STT | (tool: `azure_stt`) | Optional cloud speech-to-text, word-level timestamps — preferred when `AZURE_SPEECH_KEY` is set | `azure-speech-to-text` |
 | Subtitle Sync | `core/subtitle-sync.md` | Subtitle timing and alignment | `remotion-best-practices` |
 | Color Grading | `core/color-grading.md` | FFmpeg color profiles, LUT workflow, accessibility | `ffmpeg` |

@@ -306,7 +307,7 @@ Claude Code accesses them via symlinks in `.claude/skills/`.
 |----------|-----------------|--------|
 | **Video Composition** | `remotion-best-practices`, `remotion`, `hyperframes` (router), `hyperframes-core`, `hyperframes-creative`, `hyperframes-media`, `hyperframes-animation`, `hyperframes-cli`, `hyperframes-registry`, `media-use`, `motion-graphics`, `music-to-video`, `remotion-to-hyperframes`, `website-to-video` | `remotion-dev/skills`, `digitalsamba/claude-code-video-toolkit`, `heygen-com/hyperframes` (vendored v0.7.17, see `.agents/skills/hyperframes/PROVENANCE.md`) |
 | **Video Processing** | `ffmpeg`, `video-toolkit` | `digitalsamba/claude-code-video-toolkit` |
-| **TTS & Audio** | `text-to-speech`, `speech-to-text`, `music`, `sound-effects`, `elevenlabs`, `agents`, `setup-api-key` | `elevenlabs/skills`, `digitalsamba/claude-code-video-toolkit` |
+| **TTS & Audio** | `text-to-speech`, `speech-to-text` (whisper, default STT), `azure-speech-to-text` (optional cloud STT), `music`, `sound-effects`, `elevenlabs`, `agents`, `setup-api-key` | `elevenlabs/skills`, `digitalsamba/claude-code-video-toolkit` |
 | **Image Generation** | `flux-best-practices`, `bfl-api`, `grok-media` | `black-forest-labs/skills`, local OpenMontage skill |
 | **Math Animation** | `manimce-best-practices`, `manimgl-best-practices`, `manim-composer` | `adithya-s-k/manim_skill` |
 | **3D Graphics** | `threejs-animation`, `threejs-fundamentals`, `threejs-geometry`, `threejs-interaction`, `threejs-lighting`, `threejs-loaders`, `threejs-materials`, `threejs-postprocessing`, `threejs-shaders`, `threejs-textures` | `cloudai-x/threejs-skills` |
--- a/tests/tools/test_azure_stt.py
+++ b/tests/tools/test_azure_stt.py
@@ -0,0 +1,242 @@
+"""Focused tests for the Azure AI Speech (Fast Transcription) STT tool.
+
+No live API calls: the network layer is monkeypatched. Covers the tool
+contract, registry discovery, status behavior, the response→transcript
+mapping, execute() guardrails, and downstream subtitle compatibility.
+"""
+
+import sys
+from pathlib import Path
+
+import pytest
+
+PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
+sys.path.insert(0, str(PROJECT_ROOT))
+
+from tools.base_tool import BaseTool, ToolStatus, ToolTier, ToolRuntime
+from tools.tool_registry import ToolRegistry
+from tools.analysis.azure_stt import AzureSpeechToText
+from tools.subtitle.subtitle_gen import SubtitleGen
+
+
+# A representative Fast Transcription response payload.
+SAMPLE_PAYLOAD = {
+    "durationMilliseconds": 2400,
+    "combinedPhrases": [{"text": "Hello world this is a test"}],
+    "phrases": [
+        {
+            "offsetMilliseconds": 0,
+            "durationMilliseconds": 1200,
+            "text": "Hello world",
+            "locale": "en-US",
+            "confidence": 0.97,
+            "speaker": 1,
+            "words": [
+                {"text": "Hello", "offsetMilliseconds": 0, "durationMilliseconds": 500},
+                {"text": "world", "offsetMilliseconds": 500, "durationMilliseconds": 700},
+            ],
+        },
+        {
+            "offsetMilliseconds": 1200,
+            "durationMilliseconds": 1200,
+            "text": "this is a test",
+            "locale": "en-US",
+            "confidence": 0.95,
+            "words": [
+                {"text": "this", "offsetMilliseconds": 1200, "durationMilliseconds": 300},
+                {"text": "is", "offsetMilliseconds": 1500, "durationMilliseconds": 200},
+                {"text": "a", "offsetMilliseconds": 1700, "durationMilliseconds": 200},
+                {"text": "test", "offsetMilliseconds": 1900, "durationMilliseconds": 500},
+            ],
+        },
+    ],
+}
+
+
+class _FakeResponse:
+    def __init__(self, payload, status_code=200, text="ok"):
+        self._payload = payload
+        self.status_code = status_code
+        self.text = text
+
+    def json(self):
+        return self._payload
+
+
+@pytest.fixture
+def azure_env(monkeypatch):
+    monkeypatch.setenv("AZURE_SPEECH_KEY", "fake-key")
+    monkeypatch.setenv("AZURE_SPEECH_REGION", "eastus")
+
+
+# ---- Contract ----
+
+class TestContract:
+    def test_inherits_base_tool(self):
+        assert issubclass(AzureSpeechToText, BaseTool)
+
+    def test_identity(self):
+        t = AzureSpeechToText()
+        assert t.name == "azure_stt"
+        assert t.capability == "analysis"
+        assert t.provider == "azure"
+        assert t.runtime == ToolRuntime.API
+        assert t.tier == ToolTier.CORE
+        assert t.fallback == "transcriber"
+        assert "azure-speech-to-text" in t.agent_skills
+        assert len(t.capabilities) > 0
+
+    def test_get_info_valid(self):
+        info = AzureSpeechToText().get_info()
+        assert info["name"] == "azure_stt"
+        assert info["capability"] == "analysis"
+        assert "input_path" in info["input_schema"]["properties"]
+
+    def test_estimate_cost_by_duration(self):
+        t = AzureSpeechToText()
+        assert t.estimate_cost({"duration_seconds": 3600}) == pytest.approx(1.0)
+        assert t.estimate_cost({}) == 0.0
+
+
+# ---- Registry discovery ----
+
+class TestDiscovery:
+    def test_discoverable(self):
+        reg = ToolRegistry()
+        reg.discover("tools")
+        assert reg.get("azure_stt") is not None
+
+    def test_capability_routing(self):
+        reg = ToolRegistry()
+        reg.discover("tools")
+        names = [t.name for t in reg.get_by_capability("analysis")]
+        assert "azure_stt" in names
+
+
+# ---- Status behavior ----
+
+class TestStatus:
+    def test_unavailable_without_env(self, monkeypatch):
+        monkeypatch.delenv("AZURE_SPEECH_KEY", raising=False)
+        monkeypatch.delenv("AZURE_SPEECH_REGION", raising=False)
+        monkeypatch.delenv("AZURE_SPEECH_ENDPOINT", raising=False)
+        assert AzureSpeechToText().get_status() == ToolStatus.UNAVAILABLE
+
+    def test_available_with_key_and_region(self, azure_env):
+        assert AzureSpeechToText().get_status() == ToolStatus.AVAILABLE
+
+    def test_available_with_key_and_endpoint(self, monkeypatch):
+        monkeypatch.setenv("AZURE_SPEECH_KEY", "fake-key")
+        monkeypatch.delenv("AZURE_SPEECH_REGION", raising=False)
+        monkeypatch.setenv("AZURE_SPEECH_ENDPOINT", "https://custom.example.com")
+        assert AzureSpeechToText().get_status() == ToolStatus.AVAILABLE
+
+    def test_key_alone_is_not_enough(self, monkeypatch):
+        monkeypatch.setenv("AZURE_SPEECH_KEY", "fake-key")
+        monkeypatch.delenv("AZURE_SPEECH_REGION", raising=False)
+        monkeypatch.delenv("AZURE_SPEECH_ENDPOINT", raising=False)
+        assert AzureSpeechToText().get_status() == ToolStatus.UNAVAILABLE
+
+
+# ---- Response → transcript mapping (the risky logic) ----
+
+class TestParsePayload:
+    def test_maps_segments_words_and_timestamps(self):
+        data = AzureSpeechToText._parse_payload(SAMPLE_PAYLOAD, ["en-US"])
+        assert data["language"] == "en-US"
+        assert data["duration_seconds"] == 2.4
+        assert len(data["segments"]) == 2
+
+        seg0 = data["segments"][0]
+        assert seg0["text"] == "Hello world"
+        assert seg0["start"] == 0.0 and seg0["end"] == 1.2
+        assert seg0["speaker"] == 1
+        # ms → seconds conversion on words
+        assert seg0["words"][0] == {
+            "word": "Hello", "start": 0.0, "end": 0.5, "probability": 0.97,
+        }
+        # flat word stream aggregates every phrase
+        assert len(data["word_timestamps"]) == 6
+
+    def test_locale_resolution(self):
+        r = AzureSpeechToText._resolve_locales
+        assert r({"language": "en"}) == ["en-US"]        # ISO → default locale
+        assert r({"language": "en-GB"}) == ["en-GB"]     # explicit locale kept
+        assert r({"candidate_locales": ["fr-FR"]}) == ["fr-FR"]
+        assert len(r({})) > 1                             # falls back to a shortlist
+
+    def test_duration_falls_back_to_last_segment(self):
+        payload = {k: v for k, v in SAMPLE_PAYLOAD.items() if k != "durationMilliseconds"}
+        data = AzureSpeechToText._parse_payload(payload, ["en-US"])
+        assert data["duration_seconds"] == pytest.approx(2.4)
+
+    def test_output_feeds_subtitle_gen(self, tmp_path):
+        """Parsed segments must be a drop-in for the subtitle generator."""
+        data = AzureSpeechToText._parse_payload(SAMPLE_PAYLOAD, ["en-US"])
+        out = tmp_path / "captions.srt"
+        res = SubtitleGen().execute(
+            {"segments": data["segments"], "format": "srt", "output_path": str(out)}
+        )
+        assert res.success
+        assert out.exists()
+        assert "Hello world" in out.read_text()
+
+
+# ---- execute() guardrails + mocked success ----
+
+class TestExecute:
+    def test_missing_file(self, azure_env, tmp_path):
+        res = AzureSpeechToText().execute({"input_path": str(tmp_path / "nope.wav")})
+        assert not res.success
+        assert "not found" in res.error.lower()
+
+    def test_missing_credentials(self, monkeypatch, tmp_path):
+        monkeypatch.delenv("AZURE_SPEECH_KEY", raising=False)
+        monkeypatch.delenv("AZURE_SPEECH_REGION", raising=False)
+        audio = tmp_path / "a.wav"
+        audio.write_bytes(b"RIFF....")
+        res = AzureSpeechToText().execute({"input_path": str(audio)})
+        assert not res.success
+        assert "not configured" in res.error.lower()
+
+    def test_success_path_mocked(self, azure_env, tmp_path, monkeypatch):
+        import requests
+
+        captured = {}
+
+        def fake_post(url, headers=None, files=None, timeout=None):
+            captured["url"] = url
+            captured["headers"] = headers
+            return _FakeResponse(SAMPLE_PAYLOAD)
+
+        monkeypatch.setattr(requests, "post", fake_post)
+
+        audio = tmp_path / "a.wav"
+        audio.write_bytes(b"RIFF....")
+        res = AzureSpeechToText().execute(
+            {"input_path": str(audio), "language": "en", "output_dir": str(tmp_path)}
+        )
+
+        assert res.success
+        assert res.model == "azure-fast-transcription"
+        assert res.cost_usd == pytest.approx(0.0007, abs=1e-4)  # 2.4s at $1/hr
+        assert res.data["provider"] == "azure"
+        assert len(res.data["segments"]) == 2
+        # transcript JSON artifact was written
+        assert res.artifacts and Path(res.artifacts[0]).exists()
+        # correct endpoint + auth header used
+        assert "transcriptions:transcribe" in captured["url"]
+        assert captured["headers"]["Ocp-Apim-Subscription-Key"] == "fake-key"
+
+    def test_http_error_surfaced(self, azure_env, tmp_path, monkeypatch):
+        import requests
+
+        monkeypatch.setattr(
+            requests, "post",
+            lambda *a, **k: _FakeResponse(None, status_code=401, text="Unauthorized"),
+        )
+        audio = tmp_path / "a.wav"
+        audio.write_bytes(b"RIFF....")
+        res = AzureSpeechToText().execute({"input_path": str(audio), "output_dir": str(tmp_path)})
+        assert not res.success
+        assert "401" in res.error
--- a/tools/analysis/azure_stt.py
+++ b/tools/analysis/azure_stt.py
@@ -0,0 +1,365 @@
+"""Azure AI Speech transcription tool (Fast Transcription REST API).
+
+Speech-to-text with word-level timestamps and optional speaker diarization,
+served by Azure AI Speech. This is an optional cloud STT provider; when
+`AZURE_SPEECH_KEY` is configured the agent may prefer it for cloud
+transcription, while the local faster-whisper `transcriber` tool remains the
+default offline path.
+
+Uses the **Fast Transcription** REST endpoint, which accepts a local audio
+file directly (multipart upload) and returns a synchronous JSON result — no
+Azure Blob storage, SAS URLs, or async job polling required. Output is shaped
+to match `transcriber` exactly (segments + word_timestamps) so it is a
+drop-in for `subtitle_gen` and every pipeline that consumes a transcript.
+
+Docs: https://learn.microsoft.com/azure/ai-services/speech-service/fast-transcription-create
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import time
+from pathlib import Path
+from typing import Any
+
+from tools.base_tool import (
+    BaseTool,
+    Determinism,
+    ExecutionMode,
+    ResourceProfile,
+    ResumeSupport,
+    RetryPolicy,
+    ToolResult,
+    ToolRuntime,
+    ToolStability,
+    ToolStatus,
+    ToolTier,
+)
+
+# Fast Transcription API version (GA).
+_API_VERSION = "2024-11-15"
+
+# Candidate locales used for automatic language identification when the caller
+# does not pin a language. Azure Fast Transcription accepts several locales and
+# picks the best match per phrase.
+_DEFAULT_CANDIDATE_LOCALES = [
+    "en-US",
+    "es-ES",
+    "fr-FR",
+    "de-DE",
+    "it-IT",
+    "pt-BR",
+    "hi-IN",
+    "ja-JP",
+    "zh-CN",
+]
+
+# Minimal ISO 639-1 -> default BCP-47 locale map so callers can keep passing the
+# short codes the whisper transcriber accepts (e.g. "en", "es").
+_ISO_TO_LOCALE = {
+    "en": "en-US",
+    "es": "es-ES",
+    "fr": "fr-FR",
+    "de": "de-DE",
+    "it": "it-IT",
+    "pt": "pt-BR",
+    "hi": "hi-IN",
+    "ja": "ja-JP",
+    "ko": "ko-KR",
+    "zh": "zh-CN",
+    "ar": "ar-EG",
+    "ru": "ru-RU",
+    "nl": "nl-NL",
+    "tr": "tr-TR",
+    "pl": "pl-PL",
+    "id": "id-ID",
+}
+
+
+class AzureSpeechToText(BaseTool):
+    name = "azure_stt"
+    version = "0.1.0"
+    tier = ToolTier.CORE
+    capability = "analysis"
+    provider = "azure"
+    stability = ToolStability.BETA
+    execution_mode = ExecutionMode.SYNC
+    determinism = Determinism.DETERMINISTIC
+    runtime = ToolRuntime.API
+
+    # Availability is decided by get_status() (env var check), mirroring the
+    # ElevenLabs provider tool — dependencies stays empty.
+    dependencies = []
+    install_instructions = (
+        "Set your Azure AI Speech credentials:\n"
+        "  export AZURE_SPEECH_KEY=your_speech_resource_key\n"
+        "  export AZURE_SPEECH_REGION=eastus   # your Speech resource region\n"
+        "Create a Speech resource in the Azure portal "
+        "(https://portal.azure.com) — the key and region are on its "
+        "'Keys and Endpoint' page. Optionally set AZURE_SPEECH_ENDPOINT to a "
+        "full custom endpoint URL instead of the region."
+    )
+    agent_skills = ["azure-speech-to-text"]
+
+    capabilities = [
+        "transcribe",
+        "word_timestamps",
+        "diarization",
+        "language_detection",
+    ]
+
+    best_for = [
+        "Cloud transcription with word-level timestamps and no GPU",
+        "Multi-language auto-detection across a candidate locale set",
+        "Speaker diarization without a HuggingFace token",
+    ]
+    not_good_for = [
+        "Fully offline runs (use the whisper `transcriber` fallback)",
+        "Single files longer than ~2 hours (use Azure Batch transcription)",
+    ]
+
+    input_schema = {
+        "type": "object",
+        "required": ["input_path"],
+        "properties": {
+            "input_path": {"type": "string", "description": "Path to audio or video file"},
+            "language": {
+                "type": "string",
+                "description": (
+                    "ISO 639-1 code (e.g. 'en') or BCP-47 locale (e.g. 'en-US'). "
+                    "Omit for automatic language identification across common locales."
+                ),
+            },
+            "candidate_locales": {
+                "type": "array",
+                "items": {"type": "string"},
+                "description": (
+                    "BCP-47 locales to consider for language ID when `language` is "
+                    "not set. Defaults to a common-language shortlist."
+                ),
+            },
+            "diarize": {"type": "boolean", "default": False},
+            "max_speakers": {"type": "integer", "default": 4},
+            "profanity_filter": {
+                "type": "string",
+                "enum": ["None", "Masked", "Removed", "Tags"],
+                "default": "Masked",
+            },
+            "output_dir": {"type": "string", "description": "Directory for output files"},
+        },
+    }
+
+    output_schema = {
+        "type": "object",
+        "properties": {
+            "segments": {"type": "array"},
+            "word_timestamps": {"type": "array"},
+            "language": {"type": "string"},
+            "duration_seconds": {"type": "number"},
+        },
+    }
+
+    resource_profile = ResourceProfile(
+        cpu_cores=1,
+        ram_mb=256,
+        vram_mb=0,
+        disk_mb=50,
+        network_required=True,
+    )
+
+    retry_policy = RetryPolicy(
+        max_retries=2,
+        retryable_errors=["ConnectionError", "Timeout", "429", "503"],
+    )
+    resume_support = ResumeSupport.FROM_START
+    idempotency_key_fields = ["input_path", "language", "diarize"]
+    side_effects = ["writes transcript JSON to output_dir", "sends audio to Azure AI Speech"]
+    fallback = "transcriber"
+    fallback_tools = ["transcriber"]
+    user_visible_verification = [
+        "Check transcript text against source audio",
+        "Verify word timestamps align with speech",
+    ]
+
+    def get_status(self) -> ToolStatus:
+        if os.environ.get("AZURE_SPEECH_KEY") and (
+            os.environ.get("AZURE_SPEECH_REGION") or os.environ.get("AZURE_SPEECH_ENDPOINT")
+        ):
+            return ToolStatus.AVAILABLE
+        return ToolStatus.UNAVAILABLE
+
+    # Azure AI Speech Fast Transcription bills per audio-hour (Standard tier is
+    # roughly $1.00/audio-hour at time of writing).
+    COST_PER_AUDIO_HOUR = 1.0
+
+    def estimate_cost(self, inputs: dict[str, Any]) -> float:
+        # Duration is unknown until the audio is transcribed; return the actual
+        # cost by passing duration_seconds, else 0.0 as a pre-call estimate.
+        seconds = inputs.get("duration_seconds", 0) or 0
+        return round((seconds / 3600.0) * self.COST_PER_AUDIO_HOUR, 4)
+
+    def estimate_runtime(self, inputs: dict[str, Any]) -> float:
+        # Fast Transcription is well under real-time; conservative default.
+        return 20.0
+
+    def _endpoint(self) -> str:
+        endpoint = os.environ.get("AZURE_SPEECH_ENDPOINT")
+        if endpoint:
+            return endpoint.rstrip("/")
+        region = os.environ.get("AZURE_SPEECH_REGION", "").strip()
+        return f"https://{region}.api.cognitive.microsoft.com"
+
+    @staticmethod
+    def _resolve_locales(inputs: dict[str, Any]) -> list[str]:
+        language = inputs.get("language")
+        if language:
+            if "-" in language:  # already a BCP-47 locale
+                return [language]
+            return [_ISO_TO_LOCALE.get(language.lower(), f"{language.lower()}-US")]
+        candidates = inputs.get("candidate_locales")
+        return list(candidates) if candidates else list(_DEFAULT_CANDIDATE_LOCALES)
+
+    def execute(self, inputs: dict[str, Any]) -> ToolResult:
+        input_path = Path(inputs["input_path"])
+        if not input_path.exists():
+            return ToolResult(success=False, error=f"Input file not found: {input_path}")
+
+        api_key = os.environ.get("AZURE_SPEECH_KEY")
+        if not api_key or not (
+            os.environ.get("AZURE_SPEECH_REGION") or os.environ.get("AZURE_SPEECH_ENDPOINT")
+        ):
+            return ToolResult(
+                success=False,
+                error="Azure Speech is not configured. " + self.install_instructions,
+            )
+
+        start = time.time()
+        try:
+            result = self._transcribe(inputs, api_key, input_path)
+        except Exception as exc:
+            return ToolResult(success=False, error=f"Transcription failed: {exc}")
+
+        result.duration_seconds = round(time.time() - start, 2)
+        result.model = "azure-fast-transcription"
+        result.cost_usd = self.estimate_cost(
+            {"duration_seconds": result.data.get("duration_seconds", 0)}
+        )
+        return result
+
+    def _transcribe(
+        self, inputs: dict[str, Any], api_key: str, input_path: Path
+    ) -> ToolResult:
+        import requests
+
+        output_dir = Path(inputs.get("output_dir", input_path.parent))
+        diarize = inputs.get("diarize", False)
+        output_dir.mkdir(parents=True, exist_ok=True)
+
+        locales = self._resolve_locales(inputs)
+        definition: dict[str, Any] = {
+            "locales": locales,
+            "profanityFilterMode": inputs.get("profanity_filter", "Masked"),
+        }
+        if diarize:
+            definition["diarization"] = {
+                "maxSpeakers": inputs.get("max_speakers", 4),
+                "enabled": True,
+            }
+
+        url = f"{self._endpoint()}/speechtotext/transcriptions:transcribe?api-version={_API_VERSION}"
+        headers = {"Ocp-Apim-Subscription-Key": api_key}
+
+        try:
+            with open(input_path, "rb") as audio_file:
+                response = requests.post(
+                    url,
+                    headers=headers,
+                    files={
+                        "audio": (input_path.name, audio_file, "application/octet-stream"),
+                        "definition": (None, json.dumps(definition), "application/json"),
+                    },
+                    timeout=600,
+                )
+        except requests.RequestException as exc:  # network/timeout
+            return ToolResult(success=False, error=f"Azure Speech request failed: {exc}")
+
+        if response.status_code != 200:
+            detail = response.text[:500] if response.text else ""
+            return ToolResult(
+                success=False,
+                error=f"Azure Speech returned HTTP {response.status_code}: {detail}",
+            )
+
+        try:
+            payload = response.json()
+        except ValueError:
+            return ToolResult(
+                success=False,
+                error="Azure Speech returned a non-JSON response.",
+            )
+
+        result_data = self._parse_payload(payload, locales)
+        result_data["provider"] = "azure"
+        output_path = output_dir / f"{input_path.stem}_transcript.json"
+        output_path.write_text(json.dumps(result_data, indent=2), encoding="utf-8")
+
+        return ToolResult(
+            success=True,
+            data=result_data,
+            artifacts=[str(output_path)],
+        )
+
+    @staticmethod
+    def _parse_payload(payload: dict[str, Any], requested_locales: list[str]) -> dict[str, Any]:
+        """Map the Fast Transcription response onto the transcriber output schema."""
+        segments: list[dict[str, Any]] = []
+        word_timestamps: list[dict[str, Any]] = []
+        detected_locale: str | None = None
+
+        for idx, phrase in enumerate(payload.get("phrases", [])):
+            offset_ms = phrase.get("offsetMilliseconds", 0)
+            dur_ms = phrase.get("durationMilliseconds", 0)
+            confidence = phrase.get("confidence")
+            if detected_locale is None and phrase.get("locale"):
+                detected_locale = phrase["locale"]
+
+            seg: dict[str, Any] = {
+                "id": idx,
+                "start": round(offset_ms / 1000.0, 3),
+                "end": round((offset_ms + dur_ms) / 1000.0, 3),
+                "text": (phrase.get("text") or "").strip(),
+            }
+            if "speaker" in phrase:
+                seg["speaker"] = phrase["speaker"]
+
+            words = phrase.get("words") or []
+            if words:
+                seg_words = []
+                for w in words:
+                    w_offset = w.get("offsetMilliseconds", 0)
+                    w_dur = w.get("durationMilliseconds", 0)
+                    word_entry = {
+                        "word": w.get("text", ""),
+                        "start": round(w_offset / 1000.0, 3),
+                        "end": round((w_offset + w_dur) / 1000.0, 3),
+                        # Fast Transcription has no per-word confidence; carry the
+                        # phrase confidence so downstream schemas stay populated.
+                        "probability": round(confidence, 3) if confidence is not None else None,
+                    }
+                    seg_words.append(word_entry)
+                    word_timestamps.append(word_entry)
+                seg["words"] = seg_words
+
+            segments.append(seg)
+
+        duration_ms = payload.get("durationMilliseconds")
+        if duration_ms is None and segments:
+            duration_ms = max(int(s["end"] * 1000) for s in segments)
+
+        return {
+            "segments": segments,
+            "word_timestamps": word_timestamps,
+            "language": detected_locale or (requested_locales[0] if requested_locales else None),
+            "duration_seconds": round((duration_ms or 0) / 1000.0, 3),
+        }