mirror of
https://github.com/calesthio/OpenMontage.git
synced 2026-08-05 15:20:40 +08:00
fix(video): normalize Gemini Omni file URIs; document provider in PROVIDERS.md
Review findings from PR #333: P1: _download_via_uri assumed output_video.uri is always files/<id>. The API can return a full resource URI or a ready-made .../files/<id>:download?alt=media download URL, which produced an invalid poll path with a second :download appended. New _file_id_from_uri() extracts the bare id from every documented shape; regression tests cover the full-URL form plus a parametrized matrix of URI shapes. P2: docs/PROVIDERS.md still described the Google key as TTS + Imagen only. The shared-key section now covers gemini_omni_video (model id, ~$0.10/sec pricing table, paid-tier-only, edit-turn billing note), and the env snippet, provider-to-tool mapping, and capability coverage tables include the new provider.
This commit is contained in:
@@ -33,7 +33,7 @@ PEXELS_API_KEY= # Stock photos + videos
|
||||
PIXABAY_API_KEY= # Stock photos + videos
|
||||
|
||||
# GOOGLE (one key, two tools, generous free tier)
|
||||
GOOGLE_API_KEY= # Google TTS + Google Imagen
|
||||
GOOGLE_API_KEY= # Google TTS + Google Imagen + Gemini Omni video
|
||||
|
||||
# VOICE + MUSIC
|
||||
ELEVENLABS_API_KEY= # TTS, music, sound effects (10K chars/month free)
|
||||
@@ -245,12 +245,12 @@ Doubao Speech 2.0 is billed by character package or usage in Volcengine. OpenMon
|
||||
|
||||
---
|
||||
|
||||
### Google — TTS + Imagen (Shared Key)
|
||||
### Google — TTS + Imagen + Gemini Omni Video (Shared Key)
|
||||
|
||||
> **One key, two tools.** Google Cloud TTS has 700+ voices in 50+ languages — the strongest localization option. Imagen 4 generates high-quality images.
|
||||
> **One key, three tools.** Google Cloud TTS has 700+ voices in 50+ languages — the strongest localization option. Imagen 4 generates high-quality images. Gemini Omni Flash generates and *conversationally edits* short videos — the only provider in the fleet that can refine a clip with natural language instead of regenerating it.
|
||||
|
||||
**Tools unlocked:** `google_tts`, `google_imagen`
|
||||
**Env var:** `GOOGLE_API_KEY`
|
||||
**Tools unlocked:** `google_tts`, `google_imagen`, `gemini_omni_video`
|
||||
**Env var:** `GOOGLE_API_KEY` (or `GEMINI_API_KEY` — either works; `GEMINI_API_KEY` takes precedence)
|
||||
|
||||
#### Setup
|
||||
|
||||
@@ -265,7 +265,7 @@ Doubao Speech 2.0 is billed by character package or usage in Volcengine. OpenMon
|
||||
2. Click **Enable**
|
||||
3. Make sure your API key's restrictions allow the Text-to-Speech API
|
||||
|
||||
**For Imagen**, enable the Generative Language API:
|
||||
**For Imagen and Gemini Omni video**, enable the Generative Language API:
|
||||
1. Visit [console.cloud.google.com/apis/library/generativelanguage.googleapis.com](https://console.cloud.google.com/apis/library/generativelanguage.googleapis.com)
|
||||
2. Click **Enable**
|
||||
|
||||
@@ -291,7 +291,15 @@ The free tiers apply *independently* — you get 1M Standard AND 1M WaveNet AND
|
||||
|
||||
**Free tier for Imagen:** None. Paid tier only.
|
||||
|
||||
**New account bonus:** Google Cloud offers **$300 in free credits** for new accounts (90-day trial), applicable to both TTS and Imagen.
|
||||
#### Gemini Omni Video Pricing
|
||||
|
||||
| Model | Price | Notes |
|
||||
|-------|-------|-------|
|
||||
| `gemini-omni-flash-preview` | ~$0.10 per second of video | Billed as 5,792 output tokens/sec of 720p video at $17.50/1M tokens |
|
||||
|
||||
Generates 3–10 second clips at 720p/24fps with synthesized audio, plus stateful conversational editing (`edit_video` via `previous_interaction_id`). **Paid tier only — no free tier.** A typical 8-second clip costs ~$0.80; each edit turn generates a new clip and bills again.
|
||||
|
||||
**New account bonus:** Google Cloud offers **$300 in free credits** for new accounts (90-day trial), applicable to TTS, Imagen, and Gemini Omni video.
|
||||
|
||||
#### Google TTS Voice Types
|
||||
|
||||
@@ -764,7 +772,7 @@ These tools require only FFmpeg or Python packages — no GPU, no API key.
|
||||
| **Pexels** | `PEXELS_API_KEY` | `pexels_image`, `pexels_video` | Free |
|
||||
| **Pixabay** | `PIXABAY_API_KEY` | `pixabay_image`, `pixabay_video` | Free |
|
||||
| **Piper** | — (install only) | `piper_tts` | Free |
|
||||
| **Google** | `GOOGLE_API_KEY` | `google_tts`, `google_imagen` | Free tier + paid |
|
||||
| **Google** | `GOOGLE_API_KEY` (or `GEMINI_API_KEY`) | `google_tts`, `google_imagen`, `gemini_omni_video` | Free tier (TTS) + paid (Imagen, Omni video) |
|
||||
| **ElevenLabs** | `ELEVENLABS_API_KEY` | `elevenlabs_tts`, `music_gen` | Free tier + paid |
|
||||
| **fal.ai** | `FAL_KEY` | `flux_image`, `recraft_image`, `kling_video`, `veo_video`, `minimax_video` | Pay-as-you-go |
|
||||
| **OpenAI** | `OPENAI_API_KEY` | `openai_tts`, `openai_image` | Paid only |
|
||||
@@ -786,7 +794,7 @@ How many providers cover each capability:
|
||||
| Capability | Cloud Providers | Local Providers | Free Options |
|
||||
|-----------|----------------|-----------------|--------------|
|
||||
| **Image Generation** | FLUX, Grok, Google Imagen, GPT Image 2, Recraft | Local Diffusion | Pexels, Pixabay (stock) |
|
||||
| **Video Generation** | Grok, Kling, Runway, Veo, Higgsfield, MiniMax, HeyGen | WAN, Hunyuan, CogVideo, LTX | Pexels, Pixabay (stock) |
|
||||
| **Video Generation** | Grok, Kling, Runway, Veo, Gemini Omni, Higgsfield, MiniMax, HeyGen | WAN, Hunyuan, CogVideo, LTX | Pexels, Pixabay (stock) |
|
||||
| **Text-to-Speech** | ElevenLabs, Google TTS, OpenAI | Piper | Piper, Google free tier, ElevenLabs free tier |
|
||||
| **Music Generation** | ElevenLabs, Suno | — | ElevenLabs free tier |
|
||||
| **Post-Production** | — | FFmpeg (compose, stitch, trim, mix, enhance, grade) | All free |
|
||||
|
||||
@@ -139,6 +139,47 @@ def test_gemini_omni_text_to_video_via_uri_delivery(monkeypatch, tmp_path, gemin
|
||||
assert calls["get"][1]["params"] == {"alt": "media"}
|
||||
|
||||
|
||||
def test_gemini_omni_uri_delivery_handles_full_download_url(monkeypatch, tmp_path, gemini_env):
|
||||
"""The API may return a full .../files/<id>:download?alt=media URL, not just files/<id>."""
|
||||
from tools.video.gemini_omni_video import GeminiOmniVideo
|
||||
|
||||
full_url = (
|
||||
"https://generativelanguage.googleapis.com/v1beta/files/vid-456:download?alt=media"
|
||||
)
|
||||
calls = _install_fake_requests(
|
||||
monkeypatch,
|
||||
post_responses=[FakeResponse({"id": "int_5", "output_video": {"uri": full_url}})],
|
||||
get_responses=[
|
||||
FakeResponse({"state": "ACTIVE"}),
|
||||
FakeResponse(content=b"full url mp4"),
|
||||
],
|
||||
)
|
||||
|
||||
output_path = tmp_path / "full.mp4"
|
||||
result = GeminiOmniVideo().execute({"prompt": "A sunset.", "output_path": str(output_path)})
|
||||
|
||||
assert result.success, result.error
|
||||
assert output_path.read_bytes() == b"full url mp4"
|
||||
assert calls["get"][0]["url"].endswith("/files/vid-456")
|
||||
assert calls["get"][1]["url"].endswith("/files/vid-456:download")
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"uri",
|
||||
[
|
||||
"files/vid-456",
|
||||
"files/vid-456/",
|
||||
"v1beta/files/vid-456",
|
||||
"https://generativelanguage.googleapis.com/v1beta/files/vid-456",
|
||||
"https://generativelanguage.googleapis.com/v1beta/files/vid-456:download?alt=media",
|
||||
],
|
||||
)
|
||||
def test_gemini_omni_file_id_extraction_covers_documented_uri_shapes(uri):
|
||||
from tools.video.gemini_omni_video import GeminiOmniVideo
|
||||
|
||||
assert GeminiOmniVideo._file_id_from_uri(uri) == "vid-456"
|
||||
|
||||
|
||||
def test_gemini_omni_inline_data_response_is_handled(monkeypatch, tmp_path, gemini_env):
|
||||
from tools.video.gemini_omni_video import GeminiOmniVideo
|
||||
|
||||
|
||||
@@ -296,9 +296,24 @@ class GeminiOmniVideo(BaseTool):
|
||||
return item
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _file_id_from_uri(uri: str) -> str:
|
||||
"""Extract the bare file id from any documented URI shape.
|
||||
|
||||
The API may return ``files/<id>``, a full
|
||||
``https://.../v1beta/files/<id>`` resource URI, or a ready-made
|
||||
download URL ``.../files/<id>:download?alt=media``. Polling and
|
||||
download both need just ``<id>``.
|
||||
"""
|
||||
path = uri.split("?", 1)[0].rstrip("/")
|
||||
marker = "files/"
|
||||
idx = path.rfind(marker)
|
||||
tail = path[idx + len(marker):] if idx != -1 else path.split("/")[-1]
|
||||
return tail.split(":", 1)[0]
|
||||
|
||||
def _download_via_uri(self, requests_mod: Any, api_key: str, uri: str) -> bytes:
|
||||
"""Poll a Files API entry until ACTIVE, then download its bytes."""
|
||||
file_id = uri.rstrip("/").split("/")[-1]
|
||||
file_id = self._file_id_from_uri(uri)
|
||||
headers = {"x-goog-api-key": api_key}
|
||||
deadline = time.time() + _MAX_POLL_SECONDS
|
||||
while True:
|
||||
|
||||
Reference in New Issue
Block a user