mirror of
https://github.com/calesthio/OpenMontage.git
synced 2026-08-12 11:43:54 +08:00
feat(stt): add Azure AI Speech as an optional cloud speech-to-text provider
Add an Azure AI Speech transcription tool. It is opt-in: when AZURE_SPEECH_KEY is configured the agent may prefer it for cloud STT, while the local faster-whisper `transcriber` stays the default offline path. Shared pipeline manifests are intentionally left unchanged, so no default provider selection is altered for existing users. - tools/analysis/azure_stt.py: new `azure_stt` tool (capability=analysis, provider=azure) calling the Fast Transcription REST API. The local file is uploaded via multipart and transcribed synchronously with word-level timestamps and optional diarization — no Blob storage or async polling. Output schema mirrors `transcriber` exactly, so it is a drop-in for `subtitle_gen` and other transcript consumers. Follows the existing provider-tool conventions (env-var status check, `_transcribe` helper, cost_usd/model on the result, fallback="transcriber"). - Auto-discovered by the registry; no registry or selector changes. - tests/tools/test_azure_stt.py: contract, discovery, status, response mapping, execute guardrails, and a mocked-network success path (no live API calls). - .agents/skills + .claude/skills: azure-speech-to-text Layer-3 skill. - docs/PROVIDERS.md: Azure AI Speech setup, API notes, and pricing. - .env.example, skills/INDEX.md, AGENT_GUIDE.md: document the optional cloud STT path alongside the default whisper transcriber.
This commit is contained in:
@@ -43,6 +43,10 @@ DOUBAO_SPEECH_API_KEY= # Volcengine Doubao Speech TTS (strong Mandarin nar
|
||||
DOUBAO_SPEECH_VOICE_TYPE= # Default Doubao speaker/voice type
|
||||
DASHSCOPE_API_KEY= # Alibaba DashScope (Qwen image gen, TTS, ASR with word timestamps)
|
||||
|
||||
# SPEECH-TO-TEXT (optional cloud transcription; local whisper is the default)
|
||||
AZURE_SPEECH_KEY= # Azure AI Speech — Fast Transcription (word-level timestamps)
|
||||
AZURE_SPEECH_REGION= # Speech resource region, e.g. eastus
|
||||
|
||||
# MULTI-MODEL GATEWAY (one key, 6+ tools)
|
||||
FAL_KEY= # FLUX, Recraft, Kling, Veo, MiniMax video
|
||||
|
||||
@@ -245,6 +249,54 @@ Doubao Speech 2.0 is billed by character package or usage in Volcengine. OpenMon
|
||||
|
||||
---
|
||||
|
||||
### Azure AI Speech — Speech-to-Text
|
||||
|
||||
> **Cloud transcription.** Azure AI Speech Fast Transcription turns local audio into text with word-level timestamps, speaker diarization, and multi-language identification — no GPU required. Optional: the local faster-whisper `transcriber` remains the default offline STT path. When `AZURE_SPEECH_KEY` is set, the agent prefers `azure_stt` for cloud transcription.
|
||||
|
||||
**Tools unlocked:** `azure_stt`
|
||||
**Env vars:** `AZURE_SPEECH_KEY`, `AZURE_SPEECH_REGION` (or `AZURE_SPEECH_ENDPOINT`)
|
||||
|
||||
#### Setup
|
||||
|
||||
1. In the [Azure portal](https://portal.azure.com), create a **Speech** resource (Azure AI services → Speech service).
|
||||
2. Open the resource's **Keys and Endpoint** page.
|
||||
3. Copy **KEY 1** and the **Location/Region** (e.g. `eastus`).
|
||||
4. Add to `.env`:
|
||||
```bash
|
||||
AZURE_SPEECH_KEY=your-speech-resource-key
|
||||
AZURE_SPEECH_REGION=eastus
|
||||
# AZURE_SPEECH_ENDPOINT=https://<custom>... # optional, overrides region
|
||||
```
|
||||
|
||||
#### API Notes
|
||||
|
||||
OpenMontage uses the **Fast Transcription** REST endpoint, which accepts a local
|
||||
audio file directly (multipart upload) and returns a synchronous result — no
|
||||
Azure Blob storage, SAS URLs, or async job polling:
|
||||
|
||||
```text
|
||||
POST https://{region}.api.cognitive.microsoft.com/speechtotext/transcriptions:transcribe?api-version=2024-11-15
|
||||
Ocp-Apim-Subscription-Key: ${AZURE_SPEECH_KEY}
|
||||
```
|
||||
|
||||
For files longer than ~2 hours or bulk jobs, use Azure Batch Transcription instead (not wired into OpenMontage).
|
||||
|
||||
#### What It Is Best For
|
||||
|
||||
- Cloud transcription with word-level timestamps and no local GPU
|
||||
- Multi-language auto-detection across a candidate locale set
|
||||
- Speaker diarization without a HuggingFace token
|
||||
- Subtitle timing metadata that flows straight into `subtitle_gen`
|
||||
|
||||
#### Pricing
|
||||
|
||||
Azure AI Speech Standard (S0) bills speech-to-text by audio-hour (roughly
|
||||
$1.00/audio-hour at time of writing; a free F0 tier includes a limited monthly
|
||||
allowance). OpenMontage estimates cost from the transcribed audio duration. See
|
||||
[Azure AI Speech pricing](https://azure.microsoft.com/pricing/details/cognitive-services/speech-services/) for current rates.
|
||||
|
||||
---
|
||||
|
||||
### Google — TTS + Imagen + Music + Video (Shared Key)
|
||||
|
||||
> **One key, five tools.** Google Cloud TTS has 700+ voices in 50+ languages — the strongest localization option. Imagen 4 generates high-quality images. Google Lyria generates high-quality background music. Gemini Omni Flash supports conversational video editing, and direct Veo generation covers premium short video clips.
|
||||
|
||||
Reference in New Issue
Block a user