Voice AI stopped being a research topic in 2025. Enterprise contact-center budgets are exploding post the Salesforce Agentforce and ServiceNow AI-agent rollouts. Retell, Vapi, Deepgram, ElevenLabs Conversational, and LiveKit are all in growth mode. HN "Who is hiring? July 2026" has ten separate voice-AI-engineer postings. But content on end-to-end architecture — the latency budgets, the VAD tuning, the SIP integration gotchas, the tool-calling mid-utterance — is scattered across Twitter threads and Discord logs.
This hub is for the backend or full-stack developer who has been asked to "ship a voice agent by end of quarter." It curates skills on realistic sub-800ms end-to-end architectures, voice-activity-detection thresholds that avoid clipping short answers, barge-in handling so users can interrupt the agent, telephony integration for PSTN and SIP, and the tool-calling patterns that let a voice agent look up an order mid-conversation without an awkward silence. Every skill assumes you are building a production customer-facing product, not another Vapi demo.
Who this hub is for
Backend and full-stack engineers building production voice agents at contact-center, healthcare, sales, and consumer companies.
End-to-end 800 milliseconds from end-of-user-speech to first-audio-out is the modern bar. Under 500ms feels like a real conversation. Over 1200ms feels broken. Budget: 100-200ms speech-to-text, 300-500ms LLM generation (streaming), 100-200ms text-to-speech first byte, 50-100ms network overhead. If any single stage exceeds 500ms your total will feel slow.
LiveKit is the infrastructure layer — you write more code, but you have full control and no vendor lock-in. Retell and Vapi are managed platforms — you write less code, but you accept vendor pricing and roadmap. Rule of thumb: if you need custom telephony, custom voice cloning, or on-prem deployment, use LiveKit. If you need a working voice agent in a week, use Retell or Vapi.
Detect user speech during agent playback (VAD threshold ~30dB above noise floor for 200ms continuous). Immediately stop the TTS stream, cancel the in-flight LLM generation, and start listening. The gotcha: user cough or "uh-huh" backchannel should NOT trigger a full interrupt. Use a two-tier VAD — soft interrupt (lower volume, no cancellation) vs hard interrupt (higher volume, cancel and listen).
For B2C consumer apps (web + mobile), WebRTC via LiveKit or a similar SFU is fine. For any voice agent that dials or receives real phone numbers, you need SIP trunking (Twilio, Telnyx, Signalwire). Retell and Vapi bundle SIP; LiveKit has SIP gateway plugins. Do not attempt to bridge PSTN yourself unless telephony is your business.
Yes — this is where the modern stack gets interesting. When your LLM decides to call a function, you speak a filler ("Give me one moment...") while the function executes, then continue with the result. The filler must be pre-generated (cached TTS clips) or the latency spike defeats the whole point. Both Retell and Vapi handle this natively; on LiveKit you build it yourself.
Long-tail retries and dropped calls. A voice agent that works 95% of the time still has 1 in 20 users hitting a stuttering, restarting, or dead-air experience. Every dropped call costs you a customer. Budget as much engineering time on error handling — reconnect logic, degraded-mode fallbacks, human handoff — as on the happy path.