Skip to content

Provider page updates #1165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stephenvapiai merged 3 commits into from
Aug 20, 2026
Merged

Provider page updates #1165

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
eb26f67
Refined sections
stephenvapiai Aug 20, 2026
3923587
Updated pages
stephenvapiai Aug 20, 2026
6d5276c
Merge branch 'main' into docs/provider-updates
stephenvapiai Aug 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
499 changes: 322 additions & 177 deletions fern/apis/api/ai_examples_override.yml
View file
Open in desktop

Large diffs are not rendered by default.

131 changes: 33 additions & 98 deletions fern/assistants/examples/multilingual-agent.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,8 +298,8 @@ We will be creating a dynamic multilingual customer support agent for GlobalTech
// Option 2: Google Multilingual
const googleTranscriber = {
provider: "google",
model: "latest",
language: "multilingual"
model: "gemini-2.0-flash",
language: "Multilingual"
};

// Update assistant with transcriber
Expand All @@ -320,8 +320,8 @@ We will be creating a dynamic multilingual customer support agent for GlobalTech
# Option 2: Google Multilingual
google_transcriber = {
"provider": "google",
"model": "latest",
"language": "multilingual"
"model": "gemini-2.0-flash",
"language": "Multilingual"
}

# Update assistant with transcriber
Expand Down Expand Up @@ -356,8 +356,8 @@ We will be creating a dynamic multilingual customer support agent for GlobalTech
-d '{
"transcriber": {
"provider": "google",
"model": "latest",
"language": "multilingual"
"model": "gemini-2.0-flash",
"language": "Multilingual"
}
}'
```
Expand All @@ -371,49 +371,24 @@ We will be creating a dynamic multilingual customer support agent for GlobalTech
## 4. Configure Multilingual Voice Synthesis

<Steps>
<Step title="Set up language-specific voices">
<Step title="Set up automatic language synthesis">
<Tabs>
<Tab title="Dashboard">
1. In the **Voice** section of your assistant:
2. **Provider**: Select `Azure` (best multilingual coverage)
3. **Voice**: Choose primary voice `en-US-AriaNeural` (English)
4. **Add fallback voices**:
- Spanish: `es-ES-ElviraNeural` (Spain) or `es-MX-DaliaNeural` (Mexico)
- French: `fr-FR-DeniseNeural` (France) or `fr-CA-SylvieNeural` (Canada)
5. **Alternative providers**: ElevenLabs, OpenAI, or PlayHT all support multiple languages
1. In the **Voice** section of your assistant, select **Vapi**.
2. Choose a Vapi Voice.
3. Set **Version** to `2`.
4. Leave the language unset, or set it to `auto`.

Voice fallbacks activate when synthesis fails. They do not select a different voice based on the detected language.
</Tab>
<Tab title="TypeScript (Server SDK)">
```typescript
// Multi-language voice configuration
// Vapi Voices V2 automatically selects the synthesis language.
const multilingualVoice = {
provider: "azure",
voiceId: "en-US-AriaNeural", // Primary English voice
fallbackPlan: {
voices: [
{
provider: "azure",
voiceId: "es-ES-ElviraNeural" // Spanish (Spain)
},
{
provider: "azure",
voiceId: "fr-FR-DeniseNeural" // French (France)
},
{
provider: "azure",
voiceId: "es-MX-DaliaNeural" // Spanish (Mexico)
},
{
provider: "azure",
voiceId: "fr-CA-SylvieNeural" // French (Canada)
}
]
}
};

// Alternative: ElevenLabs multilingual
const elevenLabsVoice = {
provider: "11labs",
voiceId: "multilingual-v2" // Supports multiple languages
provider: "vapi",
voiceId: "Elliot",
version: 2,
language: "auto"
};

await vapi.assistants.update("YOUR_ASSISTANT_ID", {
Expand All @@ -423,30 +398,12 @@ We will be creating a dynamic multilingual customer support agent for GlobalTech
</Tab>
<Tab title="Python (Server SDK)">
```python
# Multi-language voice configuration
# Vapi Voices V2 automatically selects the synthesis language.
multilingual_voice = {
"provider": "azure",
"voiceId": "en-US-AriaNeural", # Primary English voice
"fallbackPlan": {
"voices": [
{
"provider": "azure",
"voiceId": "es-ES-ElviraNeural" # Spanish (Spain)
},
{
"provider": "azure",
"voiceId": "fr-FR-DeniseNeural" # French (France)
},
{
"provider": "azure",
"voiceId": "es-MX-DaliaNeural" # Spanish (Mexico)
},
{
"provider": "azure",
"voiceId": "fr-CA-SylvieNeural" # French (Canada)
}
]
}
"provider": "vapi",
"voiceId": "Elliot",
"version": 2,
"language": "auto"
}

# Update assistant with voice configuration
Expand All @@ -467,20 +424,10 @@ We will be creating a dynamic multilingual customer support agent for GlobalTech
-H "Content-Type: application/json" \
-d '{
"voice": {
"provider": "azure",
"voiceId": "en-US-AriaNeural",
"fallbackPlan": {
"voices": [
{
"provider": "azure",
"voiceId": "es-ES-ElviraNeural"
},
{
"provider": "azure",
"voiceId": "fr-FR-DeniseNeural"
}
]
}
"provider": "vapi",
"voiceId": "Elliot",
"version": 2,
"language": "auto"
}
}'
```
Expand Down Expand Up @@ -1084,35 +1031,23 @@ Keep responses concise (under 50 words) while being thorough and helpful."""

## Alternative: Squad-Based Language Selection

For a more structured approach with explicit language selection, see our comprehensive [Squad-based multilingual support](../../squads/examples/multilingual-support) guide. This approach lets customers choose their language at the start of the call, then routes them to dedicated conversation paths optimized for each language.
For a more structured approach with explicit language selection, see our comprehensive [Squad-based multilingual support](/squads/examples/multilingual-support) guide. This approach lets customers choose their language at the start of the call, then routes them to dedicated conversation paths optimized for each language.

## Provider Support Summary

**Speech-to-Text (Transcription):**
- **Gladia**: Solaria, automatic language detection and code-switching.
- **Deepgram**: Nova 2, Nova 3 with "Multi" language setting
- **Google**: Latest models with "Multilingual" language setting
- **All other providers**: Single language only, no automatic detection

**Text-to-Speech (Voice Synthesis):**
- **Azure**: 400+ voices across 140+ languages (recommended for coverage)
- **ElevenLabs**: 30+ languages with premium quality
- **OpenAI**: 50+ languages with consistent quality
- **PlayHT**: 80+ languages, cost-effective
- **All providers**: Support multiple languages natively
Automatic detection and code-switching require provider-specific transcriber settings. Review [Multilingual support](/customization/multilingual) and the individual [transcriber provider pages](/providers/transcriber/overview) for current configurations.

**Language Models:**
- **All major LLMs** (GPT-4o, Claude, Gemini, Llama, etc.): Native multilingual support
For speech synthesis, confirm that the selected model and voice support every language in your assistant's system prompt. Vapi Voices V2 can select the synthesis language automatically. Other providers may require a provider-specific model, voice, or language setting.

## Next Steps

Just like that, you've built a dynamic multilingual customer support agent that automatically detects and responds in the customer's language with seamless mid-conversation language switching.

Consider reading the following guides to further enhance your multilingual implementation:

* [**Squad-based Multilingual Support**](../../squads/examples/multilingual-support) - Compare with structured language selection approach
* [**Multilingual Configuration Guide**](../../../customization/multilingual) - Learn about all multilingual configuration options
* [**Custom Tools**](../../../tools/custom-tools) - Build advanced multilingual tools and integrations
* [**Squad-based Multilingual Support**](/squads/examples/multilingual-support) - Compare with structured language selection approach
* [**Multilingual Configuration Guide**](/customization/multilingual) - Learn about all multilingual configuration options
* [**Custom Tools**](/tools/custom-tools) - Build advanced multilingual tools and integrations

<Callout>
Need help with multilingual implementation? Chat with the team on our [Discord](https://discord.com/invite/pUFNcf2WmH) or mention us on [X/Twitter](https://x.com/Vapi_AI).
Expand Down
37 changes: 26 additions & 11 deletions fern/customization/multilingual.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Set up your transcriber to automatically detect and process multiple languages.
2. Create a new assistant or edit an existing one
3. In the **Transcriber** section:
- **Provider**: Select `Deepgram` (recommended), `Google`, or `Gladia`
- **Model**: For Deepgram, choose `Nova 2` or `Nova 3`; for Google, choose `Latest`; for Gladia, choose `Solaria`
- **Model**: For Deepgram, choose `Nova 2` or `Nova 3`; for Google, choose `Gemini 2.0 Flash`; for Gladia, choose `Solaria`
- **Language / Mode**: Set `Multi` (Deepgram), `Multilingual` (Google), or choose the language you want to transcribe (Gladia)
4. **Other providers**: May require a single languages and not auto-detect
5. Click **Save** to apply the configuration
Expand All @@ -53,8 +53,8 @@ Set up your transcriber to automatically detect and process multiple languages.
// Alternative: Google for multilingual support
const googleMultilingual = {
provider: "google",
model: "latest",
language: "multilingual"
model: "gemini-2.0-flash",
language: "Multilingual"
};
```
</Tab>
Expand All @@ -78,8 +78,8 @@ Set up your transcriber to automatically detect and process multiple languages.
# Alternative: Google for multilingual support
google_multilingual = {
"provider": "google",
"model": "latest",
"language": "multilingual"
"model": "gemini-2.0-flash",
"language": "Multilingual"
}
```
</Tab>
Expand All @@ -105,8 +105,8 @@ Set up your transcriber to automatically detect and process multiple languages.
-d '{
"transcriber": {
"provider": "google",
"model": "latest",
"language": "multilingual"
"model": "gemini-2.0-flash",
"language": "Multilingual"
}
}'
```
Expand Down Expand Up @@ -198,6 +198,21 @@ Configure your assistant to use appropriate voices for each detected language.
</Tab>
</Tabs>

<Note>
Vapi Voices V2 can select the synthesis language automatically. Set `voice.provider` to `vapi`, `voice.version` to `2`, and `voice.language` to `auto`:

```json
{
"voice": {
"provider": "vapi",
"voiceId": "Elliot",
"version": 2,
"language": "auto"
}
}
```
</Note>

<Note>
**Voice Provider Support:** Unlike transcription, all major voice providers (Azure, ElevenLabs, OpenAI, etc.) support multiple languages. Azure offers the most comprehensive coverage with 400+ voices across 140+ languages.
</Note>
Expand Down Expand Up @@ -517,7 +532,7 @@ Validate your configuration with different languages and scenarios.

Now that you have multilingual support configured:

- **[Build a complete multilingual agent](../assistants/examples/multilingual-agent):** Follow our step-by-step implementation guide
- **[Custom voices](custom-voices/custom-voice):** Set up region-specific custom voices
- **[System prompting](../prompting-guide):** Design effective multilingual prompts
- **[Call analysis](../call-analysis):** Monitor language performance and usage
- **[Build a complete multilingual agent](/assistants/examples/multilingual-agent):** Follow our step-by-step implementation guide
- **[Custom voices](/customization/custom-voices/custom-voice):** Set up region-specific custom voices
- **[System prompting](/prompting-guide):** Design effective multilingual prompts
- **[Call analysis](/assistants/call-analysis):** Monitor language performance and usage
17 changes: 8 additions & 9 deletions fern/customization/provider-keys.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Provider Keys
subtitle: Bring your own API keys to Vapi.
description: Connect provider credentials in Vapi and find configuration details for transcriber, model, voice, and cloud providers.
slug: customization/provider-keys
---

Expand All @@ -11,19 +12,17 @@ No problem! You can bring your own API keys to Vapi. You can add them in the [Da

## Transcription Providers

Currently, the only available transcription provider is `deepgram`. To use a custom model, you can specify the deepgram model ID in the `transcriber.model` parameter of the [Assistant](/api-reference/assistants/create-assistant).
Choose the provider and its supported options in the assistant's `transcriber` configuration. See [Transcriber providers](/providers/transcriber/overview) and the [`transcriber` configuration](/api-reference/assistants/create#request.body.transcriber) in the API reference.

## Model Providers

We are currently have support for any OpenAI-compatible endpoint. This includes services like [OpenRouter](https://openrouter.ai/), [AnyScale](https://www.anyscale.com/), [Together AI](https://www.together.ai/), or your own server.
Choose the provider and model in the assistant's `model` configuration. See [Model providers](/providers/model/overview) and the [`model` configuration](/api-reference/assistants/create#request.body.model) in the API reference.

To use one of these providers, you can specify the `provider` and `model` in the `model` parameter of the [Assistant](/api-reference/assistants/create-assistant).

You can find more details in the [Custom LLMs](/customization/custom-llm/fine-tuned-openai-models) section of the documentation.
For an OpenAI-compatible endpoint that is not represented by a native provider, see [Custom LLMs](/customization/custom-llm/using-your-server).

## Voice Providers

All voice providers are supported. Once you've validated your API through the [Dashboard](https://dashboard.vapi.ai), any voice ID from your provider can be used in the `voice.voiceId` field of the [Assistant](/api-reference/assistants/create-assistant).
Choose the provider, voice ID, and supported options in the assistant's `voice` configuration. See [Voice providers](/providers/voice/overview) and the [`voice` configuration](/api-reference/assistants/create#request.body.voice) in the API reference.

## Cloud Providers

Expand All @@ -33,6 +32,6 @@ Cloudflare R2.

You can find more details on how to configure your Cloud Provider keys here:

* [AWS S3](/providers/cloud/s3)
* [GCP Cloud Storage](/providers/cloud/gcp)
* [Cloudflare R2](/providers/cloud/cloudflare)
- [AWS S3](/providers/cloud/s3)
- [Google Cloud Storage](/providers/cloud/gcp)
- [Cloudflare R2](/providers/cloud/cloudflare)
2 changes: 1 addition & 1 deletion fern/customization/speech-configuration.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ This plan defines the parameters for when the assistant begins speaking after th

**Audio-text based providers:**

- **Deepgram Flux**: Deepgram's latest transcriber model with built-in conversational speech recognition. Flux combines high-quality speech-to-text with native turn detection, while delivering ultra-low latency and Nova-3 level accuracy. Available in English (`flux-general-en`) and multilingual (`flux-general-multi`) variants. Supported languages for `flux-general-multi`: English (`en`), Spanish (`es`), French (`fr`), German (`de`), Hindi (`hi`), Russian (`ru`), Portuguese (`pt`), Japanese (`ja`), Italian (`it`), Dutch (`nl`).
- **Deepgram Flux**: Deepgram's latest transcriber model with built-in conversational speech recognition. Flux combines high-quality speech-to-text with native turn detection, while delivering ultra-low latency and Nova-3 level accuracy. Available in English (`flux-general-en`) and multilingual (`flux-general-multi`) variants. See [Deepgram's supported languages](/providers/transcriber/deepgram#supported-languages) for the current model and language combinations.

- **Assembly**: Transcriber that also reports end-of-turn detection. To use Assembly, choose it as your transcriber without setting a separate smart endpointing plan. As transcripts arrive, we consider the `end_of_turn` flag that Assembly sends to mark the end-of-turn, stream to the LLM, and generate a response.

Expand Down
Loading
Loading