Skip to Content
Builder Docs
Agent Design & Conversation Logic

Voice, Models & STT/TTS Selection

Choosing Pipeline vs. Realtime S2S, Production Provider Trust Policy, 3-tier override precedence, key rotation, Indian language models, sample rates, custom ambient noise, and pronunciation keyterms.

Production Provider Support & Reliability Policy

Although Talkar's settings schema lists 13 LLM, 6 Realtime, 16 TTS, 14 STT, and 4 Embedding providers, only 4 BYOK providers are production-trusted, battle-tested, and actively updated:

  • OpenAI (openai LLM / TTS / STT / Embeddings, plus openai_realtime where S2S is specifically needed)
  • Sarvam (sarvam LLM, bulbul TTS, saaras / saarika STT — top choice for Indian languages & code-mixing)
  • SmallestAI (smallest lightning_v3.1 / lightning_v3.1_pro TTS, pulse / pulse-2 STT)
  • Deepgram (deepgram nova-3 / flux STT, aura-2 / aura-1 / flux TTS)

(Plus Talkar Managed Credits on managed tiers, offering managed LLM presets default, accurate, fast, lite, zen and default voice.)
All other listed providers (ElevenLabs, Cartesia, Google/Vertex, Groq, Azure, AWS Bedrock, OpenRouter, MiniMax, etc.) are untested / legacy and carry risk of higher latency, stale model defaults, streaming chunk errors, or broken turn-taking. Do not ship them to production without thorough end-to-end call testing.

1. Pipeline Mode vs. Realtime S2S Mode

ModeServices ActiveBest WhenKey Tradeoffs
Standard Pipeline (Recommended)STT -> LLM -> TTS (+ optional extraction_llm + embeddings)Multi-node forms, strict edge transitions, Indian languages, CRM tool calling, DTMF, AMD/voicemailDeterministic prompt transitions, full Silero VAD + Smart Turn control, custom keyterm boosting, and independent zero-temperature Extraction LLM.
Realtime S2SRealtime Audio Model (openai_realtime, default gpt-realtime-2 / alloy) + companion llmFluid, ultra-low-latency freeform conversations without strict multi-step routingBYOK Realtime requires a companion llm configured alongside it for tool/classification fallbacks; provider owns audio turn-taking (bot_vad_stop_secs lowered to 0.5s); automatically disables workflow voicemail detection, and complex node-to-node edge routing is harder to keep deterministic.

2. Trusted Production Providers Deep Dive

Large Language Models (LLM)

Talkar separates Conversation LLM (drives live replies & tool calls) from Extraction LLM (extraction_llm, forced to temperature = 0.0 at runtime for reliable JSON variable capture; falls back to Conversation LLM if unset).

  • OpenAI (openai) — Tier 1 Default: Supports gpt-4.1, gpt-4.1-mini (recommended balance of speed & tool accuracy), gpt-4.1-nano, and gpt-5*. For reasoning models (gpt-5*, o1*, o3*, o4*), Talkar automatically maps max_tokens to max_completion_tokens and strips unsupported sampling params (temperature, top_p, penalties); specifically on gpt-5* models, if reasoning_effort is unset, Talkar defaults reasoning_effort = "minimal" and verbosity = "low" for low voice latency.
  • Sarvam (sarvam) — Tier 1 Indian / Multilingual: Models sarvam-30b (default), sarvam-105b, sarvam-105b-conversations, with wiki_grounding and reasoning_effort options.

Speech-to-Text (STT / Transcriber)

  • Deepgram (deepgram) — Tier 1:
    • nova-3-general / nova-3-medical (80+ languages + multi): Uses Talkar's local Silero VAD + turn strategies. Supports smart_format, punctuate, numerals, endpointing (100ms), and up to 100 keyterms.
    • flux-general-en / flux-general-multi: Uses Deepgram external turn ownership (eot_threshold, eot_timeout_ms, eager_eot_threshold), bypassing Silero VAD turn-stop timers.
  • Sarvam (sarvam) — Tier 1 Indian & Code-Mixed:
    • Models: saaras:v4 (default, 23 Indian languages, supports up to 50 keyterms), saaras:v3, saarika:v2.5.
    • Modes (v3/v4): transcribe (default), translate, verbatim, translit, codemix (great for Hinglish/Tanglish callers).
    • Provider VAD sensitivity defaults (saaras:v3/v4): high_vad_sensitivity = false, start_speech_volume_threshold = -30.0 dB, positive_speech_threshold = 0.7, min_speech_frames = 3.
  • SmallestAI (smallest) — Tier 1:
    • Models: pulse (38 languages) and pulse-2 (English-only; Talkar forces English runtime + emotion_detection / gender_detection). Supports ITN normalization, eou_timeout_ms (480ms default), and weighted KEYWORD:BOOST terms (up to 100 terms).
  • OpenAI (openai): gpt-4o-transcribe.

Text-to-Speech (TTS / Voices)

  • Deepgram (deepgram): aura-2-* / aura-1-* / flux-* (model inferred from voice ID), speed (0.5–1.5), mip_opt_out privacy flag, live voice discovery supported.
  • Sarvam (sarvam): Explicitly choose bulbul:v4-flash (200+ <name>_<lang>_<style> personas, default fallback shubh_enhi_ads) or bulbul:v3 (shubh + 39 voices) rather than leaving schema default bulbul:v2 (anushka, hi-IN). Supports 12 Indian locales (hi-IN, en-IN, ta-IN, te-IN, kn-IN, ml-IN, mr-IN, bn-IN, gu-IN, pa-IN, od-IN, as-IN), speed (0.5–2.0), min_buffer_size (50), max_chunk_length (150), and v3/v4 temperature (vs. v2 pitch/loudness).
  • SmallestAI (smallest): lightning_v3.1 & lightning_v3.1_pro across 16 languages (en, hi, ta, mr, bn, gu, kn, etc.).
  • OpenAI (openai): gpt-4o-mini-tts (alloy etc.).
  • Live Voice Catalog API: Live remote voice discovery supports 6 TTS providers (elevenlabs, deepgram, sarvam, cartesia, rime, and managed voice); smallest and openai use static/manual voice IDs.

RAG Embeddings

Knowledge Base Chunked mode requires 1536-dimensional embeddings (openai text-embedding-3-small recommended). If no embedding key is configured, use full_document KB mode instead.

3. 3-Tier Configuration Precedence, Overrides & Key Rotation

Talkar resolves active AI models in strict 3-tier precedence:

  1. Per-Workflow Full Override (Workflow v2 configuration override — replaces the org stack cleanly for that workflow)
  2. Per-Workflow Partial Override (Legacy model_overrides — merges field-by-field onto the org config and inherits org API keys when omitted)
  3. Organization Default Model Configuration (Org Model Configurations v2)
  • QA Node Overrides: A QA Analysis node defaults to reusing the workflow LLM (qa_use_workflow_llm = true), or can specify its own provider/model/key when set to false.
  • Multi-Key Rotation: Any api_key field accepts either a single key string or an array of keys (["sk-1", "sk-2"]) — Talkar picks one at random per call to distribute rate limits.
  • Write-Only Masked Secrets: Existing API keys come back masked on read; leave masked untouched when editing other fields, or replace completely when rotating.

Save-Time Key Validation Reality Check

  • Live-checked when saving: OpenAI, Deepgram, Groq, and managed keys are verified via network call on save.
  • Pass-through on save (NO live probe): Sarvam and SmallestAI (along with most Tier-2 providers) return valid without hitting the provider's API when saving settings. Always place a quick test call after saving new Sarvam or SmallestAI keys.

4. Sample Rates, Ambient Audio & Dictionary Boosting

  • Sample Rates: Browser WebRTC runs at 16,000 Hz; PSTN telephony runs at 8,000 Hz (except Vonage at 16,000 Hz). Talkar caps internal audio at 16 kHz so Silero VAD runs cleanly without extra resampling overhead.
  • Ambient Background Audio: Toggle office ambience (volume default 0.3) in Workflow Settings, or upload a custom ambient .wav file per workflow to mask digital line dead-air cleanly.
  • Pronunciation / Recognition Dictionary (Configurations -> Dictionary):
    • Enter comma-separated brand names, SKUs, acronyms, or Indian names (Total Ingredients, Nutraceutical, B2B) — newlines alone do not split terms.
    • boost_affirmations (on by default) adds ["yes", "no", "yeah", "correct", "yep", "nope", "sure", "okay"].
    • Provider caps to watch: Deepgram caps merged terms at 100; Sarvam caps at 50 terms (64 chars max each, only active on saaras:v4); SmallestAI merges affirmations + STT keywords (not the workflow dictionary) up to 100 weighted terms.

Nodes, Prompts & Variables

Complete builder reference for all 9 node types, edges, built-in tools, RECORDING_ID audio, voice prompting rules, Unified Built-In Variables Cheat Sheet, extraction, and publish validation.

Turns, Silero VAD & Interruptions

Plain-English UI guide for Voice Sensitivity presets, Start/Stop turn strategies, Silero VAD vs. Provider VAD, the 3 user muting layers, Smart Turn, and Realtime S2S turn ownership.

On this page

1. Pipeline Mode vs. Realtime S2S Mode2. Trusted Production Providers Deep DiveLarge Language Models (LLM)Speech-to-Text (STT / Transcriber)Text-to-Speech (TTS / Voices)RAG Embeddings3. 3-Tier Configuration Precedence, Overrides & Key Rotation4. Sample Rates, Ambient Audio & Dictionary Boosting