Latency bake-off
Published p50 / p95 targets — not marketing fog
Every Test Live and production turn records STT, LLM TTFT, and TTS TTFB. Targets overlay live aggregates from the last 30 days by pipeline mode (carrier RTT excluded).
| Mode | Target p50 | Live p50 | Live p95 | Stack |
|---|---|---|---|---|
| Hyper | 480 ms | 440 ms | 620 ms | Deepgram nova-2-phonecall → Groq 8B instant → Cartesia Sonic 3.5 |
| Speed | 380 ms | 360 ms | 520 ms | Deepgram nova-2-phonecall → Groq Llama 3.1 8B → Cartesia Sonic |
| Smart | 650 ms | 610 ms | 890 ms | Deepgram nova-3 → Groq Llama 3.3 70B → Cartesia Sonic |
| Ultra | 320 ms | 300 ms | 480 ms | Gemini Live speech-to-speech |
- Hyper: Default bake-off mode — sub-500ms p50 vs Vapi/Retell. retell_fast VAD + 70ms filler gate.
- Speed: Lowest cascaded turn target. Presence Buffer arms at ~120ms.
- Smart: Higher accuracy managed stack. Presence Buffer ~180ms.
- Ultra: S2S path. Mid-call tools stronger on Speed/Smart/Custom.