diff --git a/agents/SKILL.md b/agents/SKILL.md index d6a8d25..d59adbd 100644 --- a/agents/SKILL.md +++ b/agents/SKILL.md @@ -152,7 +152,7 @@ function App({ signedUrl }: { signedUrl: string }) { | Provider | Models | |----------|--------| -| OpenAI | `gpt-5.4`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` | +| OpenAI | `gpt-5.5`, `gpt-5.4`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` | | Anthropic | `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` | | Google | `gemini-3.1-flash-lite-preview`, `gemini-3.1-pro-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` | | ElevenLabs | `glm-45-air-fp8`, `qwen3-30b-a3b`, `qwen35-35b-a3b`, `qwen35-397b-a17b`, `gpt-oss-120b` | diff --git a/agents/references/agent-configuration.md b/agents/references/agent-configuration.md index bff6db1..92501f7 100644 --- a/agents/references/agent-configuration.md +++ b/agents/references/agent-configuration.md @@ -197,7 +197,7 @@ to resolve per-environment auth connections at runtime. | Provider | Model IDs | |----------|-----------| -| OpenAI | `gpt-5`, `gpt-5.4`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` | +| OpenAI | `gpt-5.5`, `gpt-5.4`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` | | Anthropic | `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` | | Google | `gemini-3.1-flash-lite-preview`, `gemini-3.1-pro-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` | | ElevenLabs | `glm-45-air-fp8`, `qwen3-30b-a3b`, `qwen35-35b-a3b`, `qwen35-397b-a17b`, `gpt-oss-120b` (hosted, ultra-low latency) | @@ -244,7 +244,8 @@ platform_settings={ "call_limits": { "agent_concurrency_limit": 10, "daily_limit": 100 - } + }, + "trust_context": "low" } ``` @@ -258,6 +259,7 @@ platform_settings={ | `call_limits` | object | Concurrency and daily usage limits | | `guardrails` | object | Built-in safety and policy controls for agent interactions | | `privacy` | object | Recording, retention, and conversation history redaction settings | +| `trust_context` | string | Trust classification for the agent: `unknown`, `low`, or `high` | ### auth @@ -361,6 +363,10 @@ Use `platform_settings.widget` to configure the hosted widget and shareable page | `monitoring_enabled` | bool | `false` | Enable real-time WebSocket monitoring | | `client_events` | array | - | Client events forwarded to the connected application | | `monitoring_events` | array | - | Events forwarded to monitoring WebSocket connections | +| `source_attribution` | bool | `false` | Instructs the LLM to report sources used when knowledge base content is present | + +Common client events include `agent_response_complete`, which fires when the agent is done +responding and must be enabled in `client_events`. **file_input:** @@ -406,6 +412,9 @@ agent = client.conversational_ai.agents.create( `rag.embedding_model` supports `e5_mistral_7b_instruct`, `multilingual_e5_large_instruct`, and `qwen3_embedding_4b`. +Set `conversation_config.conversation.source_attribution` to `true` when you want the agent to +report which knowledge base sources it used in responses. + ## CRUD Operations ### Using CLI (Recommended) diff --git a/agents/references/client-tools.md b/agents/references/client-tools.md index 6f4fe1d..8c8ce62 100644 --- a/agents/references/client-tools.md +++ b/agents/references/client-tools.md @@ -177,9 +177,14 @@ context as a lazily evaluated JSON history object with user, agent, and tool ent | `disable_interruptions` | bool | `false` | Prevent user interruptions during tool execution | | `execution_mode` | string | `"immediate"` | `immediate`, `post_tool_speech`, or `async` | | `tool_call_sound` | string | - | Sound during execution: `typing`, `elevator1`-`elevator4` | -| `force_pre_tool_speech` | bool | `false` | Force agent to speak before executing tool | +| `pre_tool_speech` | string | `"auto"` | Controls whether the agent speaks before execution: `auto`, `force`, or `off` | +| `force_pre_tool_speech` | bool | `false` | Deprecated; use `pre_tool_speech` instead | | `tool_error_handling_mode` | string | `"auto"` | `auto`, `summarized`, `passthrough`, or `hide` | +MCP server configuration supports the same `pre_tool_speech`, `execution_mode`, and +`response_timeout_secs` controls at the server level, with per-tool overrides in +`tool_config_overrides`. MCP timeouts default to 30 seconds and must be 5-120 seconds. + **Note:** The default `api_schema.method` is `GET`. Always set `"method": "POST"` explicitly for webhook tools that send request bodies. ### Server Implementation (Node.js) @@ -507,7 +512,7 @@ app.post("/webhook/lookup_order", async (req, res) => { ### Timeouts -Set reasonable timeouts for webhooks using `response_timeout_secs` (5-120 seconds, default 20): +Set reasonable timeouts for webhooks using `response_timeout_secs` (5-120 seconds, default 20). MCP server tool calls use the same field with a 30-second default: ```python { diff --git a/agents/references/installation.md b/agents/references/installation.md index 49da115..8f47ea6 100644 --- a/agents/references/installation.md +++ b/agents/references/installation.md @@ -42,7 +42,7 @@ elevenlabs agents push For programmatic access and client-side integration: ```bash -npm install @elevenlabs/elevenlabs-js +npm install @elevenlabs/elevenlabs-js@latest ``` > **Important:** Always use `@elevenlabs/elevenlabs-js`. The old `elevenlabs` npm package (v1.x) is deprecated and should not be used. @@ -67,11 +67,11 @@ If you have old packages installed, remove them: npm uninstall elevenlabs # Install the current packages -npm install @elevenlabs/elevenlabs-js +npm install @elevenlabs/elevenlabs-js@latest # For browser apps, install the package that matches your UI layer: -npm install @elevenlabs/client # Vanilla JavaScript in the browser -npm install @elevenlabs/react # React on the web +npm install @elevenlabs/client@latest # Vanilla JavaScript in the browser +npm install @elevenlabs/react@latest # React on the web ``` **Import changes:** diff --git a/speech-to-text/SKILL.md b/speech-to-text/SKILL.md index 4f63357..dbf11b2 100644 --- a/speech-to-text/SKILL.md +++ b/speech-to-text/SKILL.md @@ -192,6 +192,8 @@ async def transcribe_realtime(): async with client.speech_to_text.realtime.connect( model_id="scribe_v2_realtime", include_timestamps=True, + keyterms=["ElevenLabs", "Scribe"], + no_verbatim=True, ) as connection: await connection.stream_url("https://example.com/audio.mp3") @@ -215,6 +217,8 @@ function TranscriptionComponent() { const scribe = useScribe({ modelId: "scribe_v2_realtime", commitStrategy: CommitStrategy.VAD, // Auto-commit on silence for mic input + keyterms: ["ElevenLabs", "Scribe"], + noVerbatim: true, onPartialTranscript: (data) => console.log("Partial:", data.text), onCommittedTranscript: (data) => setTranscript((prev) => prev + data.text), }); @@ -247,6 +251,8 @@ import { useScribe, CommitStrategy } from "@elevenlabs/react"; const scribe = useScribe({ modelId: "scribe_v2_realtime", commitStrategy: CommitStrategy.VAD, + keyterms: ["ElevenLabs", "Scribe"], + noVerbatim: true, // Optional VAD tuning: vadSilenceThresholdSecs: 1.5, vadThreshold: 0.4, @@ -257,6 +263,8 @@ const scribe = useScribe({ // JavaScript client: pass vad config on connect const connection = await client.speechToText.realtime.connect({ modelId: "scribe_v2_realtime", + keyterms: ["ElevenLabs", "Scribe"], + noVerbatim: true, vad: { silenceThresholdSecs: 1.5, threshold: 0.4, diff --git a/speech-to-text/references/installation.md b/speech-to-text/references/installation.md index 690d343..817b81e 100644 --- a/speech-to-text/references/installation.md +++ b/speech-to-text/references/installation.md @@ -3,7 +3,7 @@ ## JavaScript / TypeScript ```bash -npm install @elevenlabs/elevenlabs-js +npm install @elevenlabs/elevenlabs-js@latest ``` > **Important:** Always use `@elevenlabs/elevenlabs-js`. The old `elevenlabs` npm package (v1.x) is deprecated and should not be used. @@ -28,11 +28,11 @@ If you have old packages installed, remove them: npm uninstall elevenlabs # Install the current packages -npm install @elevenlabs/elevenlabs-js +npm install @elevenlabs/elevenlabs-js@latest # For client-side/browser usage, also install: -npm install @elevenlabs/client # Browser client -npm install @elevenlabs/react # React hooks +npm install @elevenlabs/client@latest # Browser client +npm install @elevenlabs/react@latest # React hooks ``` **Import changes:** @@ -45,7 +45,7 @@ import { useScribe } from "@elevenlabs/react"; ## Python ```bash -pip install elevenlabs +pip install --upgrade elevenlabs ``` ```python diff --git a/speech-to-text/references/realtime-client-side.md b/speech-to-text/references/realtime-client-side.md index de169fb..67aa169 100644 --- a/speech-to-text/references/realtime-client-side.md +++ b/speech-to-text/references/realtime-client-side.md @@ -6,10 +6,10 @@ Stream audio from the browser directly to ElevenLabs for real-time transcription ```bash # React -npm install @elevenlabs/react @elevenlabs/elevenlabs-js +npm install @elevenlabs/react@latest @elevenlabs/elevenlabs-js@latest # JavaScript -npm install @elevenlabs/client @elevenlabs/elevenlabs-js +npm install @elevenlabs/client@latest @elevenlabs/elevenlabs-js@latest ``` > **Warning:** Always use the `@elevenlabs/*` namespace for client-side packages. @@ -118,6 +118,8 @@ async function startTranscription() { token, modelId: "scribe_v2_realtime", includeTimestamps: true, + keyterms: ["ElevenLabs", "Scribe"], + noVerbatim: true, microphone: { echoCancellation: true, noiseSuppression: true, @@ -155,6 +157,8 @@ async function startTranscription() { } ``` +`keyterms` biases realtime recognition toward important terms. `noVerbatim` removes filler words, false starts, and disfluencies from committed transcripts. + ## Manual Audio Chunking For file uploads or custom audio sources, encode to PCM-16 and send in chunks: diff --git a/speech-to-text/references/realtime-server-side.md b/speech-to-text/references/realtime-server-side.md index 80706b4..2e70c51 100644 --- a/speech-to-text/references/realtime-server-side.md +++ b/speech-to-text/references/realtime-server-side.md @@ -6,10 +6,10 @@ Transcribe audio streams in real-time from your server with ultra-low latency. ```bash # Python -pip install elevenlabs python-dotenv pydub +pip install --upgrade elevenlabs python-dotenv pydub # JavaScript -npm install @elevenlabs/elevenlabs-js dotenv +npm install @elevenlabs/elevenlabs-js@latest dotenv ``` > **Warning:** Do not use `npm install elevenlabs` - that's an outdated v1.x package. Always use `@elevenlabs/elevenlabs-js`. @@ -43,6 +43,8 @@ async def main(): model_id="scribe_v2_realtime", url="https://npr-ice.streamguys1.com/live.mp3", include_timestamps=True, + keyterms=["ElevenLabs", "Scribe"], + no_verbatim=True, )) def on_partial_transcript(data): @@ -86,6 +88,8 @@ const connection = await elevenlabs.speechToText.realtime.connect({ modelId: "scribe_v2_realtime", url: "https://npr-ice.streamguys1.com/live.mp3", includeTimestamps: true, + keyterms: ["ElevenLabs", "Scribe"], + noVerbatim: true, }); connection.on(RealtimeEvents.PARTIAL_TRANSCRIPT, (transcript) => { @@ -286,6 +290,13 @@ For cases where the SDK cannot be used: wss://api.elevenlabs.io/v1/speech-to-text/realtime?model_id=scribe_v2_realtime ``` +Add repeated `keyterms` query parameters to bias recognition toward specific terms, and set +`no_verbatim=true` to remove filler words, false starts, and disfluencies: + +``` +wss://api.elevenlabs.io/v1/speech-to-text/realtime?model_id=scribe_v2_realtime&keyterms=ElevenLabs&keyterms=Scribe&no_verbatim=true +``` + ### Message Format ```json