The Challenge of Real-Time Voice AI in Companion Apps

When Character.AI opened its voice call feature to users on June 27, 2024, the company reported more than 20 million calls from over 3 million unique users during the early rollout — proof that people don't just want to talk to their AI companions, they want it to feel like a real conversation. Making that happen, though, is one of the hardest engineering problems in the category.

The core challenge is latency. For voice to feel natural rather than push-to-talk, the system needs to respond in under 200 milliseconds, fast enough for the kind of turn-taking humans do without thinking. As OpenAI's engineers put it, "Voice AI only feels natural if conversation moves at the speed of speech" — when the network lags, users hear it immediately as awkward pauses, clipped interruptions, or delayed interruptions. That's why the industry has converged on WebRTC-based architectures with streaming audio, where transcription, reasoning, and speech generation all begin while the user is still talking, supported by globally distributed relay infrastructure.

Companion apps face a second, harder layer: the voice has to carry emotion. A companion-focused voice analysis notes these apps need to handle "long, emotionally varied conversations at consumer-scale economics" — including emotional expressiveness, non-verbal audio cues like laughter or sighs, and a cost per user that doesn't sink the business. That trade-off is real: the fastest speech models achieve time-to-first-audio of just 40ms but with a narrower emotional range, while more expressive voices cost more per user. For apps where the user base is mostly free and highly engaged, cost per user is a make-or-break metric.

There's also the engagement question. Voice is still the minority experience in the category — text-based companions held 45%+ market share in 2024 — but multi-modal platforms that blend text, voice, and image are the fastest-growing segment, and social companionship is the largest application category at 29.4% share in 2025. Getting voice right matters because it deepens exactly the emotional bond that drives retention.

Finally, there's the infrastructure and safety burden. Serving real-time voice at consumer scale requires the same kind of geo-distributed relay networks that power systems serving hundreds of millions of users, plus age-verification and moderation — pressure that's now formal: the FTC issued 6(b) inquiry orders to seven companies including Character Technologies, examining how companion chatbots affect younger users.

This is the bar FlirtAlly is building toward: real-time in-app calls with assigned, consistent voices per companion, plus call transcripts that flow back into long-term memory so the conversation continues after you hang up. The technical hurdles are significant — but so is the payoff when a voice call actually feels like talking to a person.

• Sub-200ms latency is the threshold for natural turn-taking
• Emotional expressiveness and non-verbal audio matter as much as speed
• Streaming pipelines let the AI start responding while you're still speaking
• Voice must work at consumer-scale economics, not enterprise pricing

Industry-Standard Tech Behind Natural Voice Chat

When Character.AI launched voice calls in June 2024, users placed more than 20 million calls from over 3 million unique accounts in the early rollout alone — proof that talking to an AI is no longer a gimmick. But what actually makes a voice conversation feel natural rather than like a laggy walkie-talkie? The answer lies in a stack of engineering that the industry has quietly standardized.

No public source documents Character.AI's internal voice pipeline, so the clearest blueprint comes from OpenAI's published engineering account of running real-time voice at scale. The foundation is WebRTC, the browser and mobile protocol that handles the unglamorous plumbing of live audio: NAT traversal, DTLS/SRTP encryption, codec negotiation, and quality monitoring. Your phone's echo cancellation and jitter buffering happen client-side, so the audio you hear stays intelligible even on imperfect networks.

The second pillar is streaming. Instead of waiting for you to finish a sentence, a spoken agent can begin transcribing, reasoning, and generating speech while you are still talking. As OpenAI's engineers put it, that is "the difference between a system that feels conversational and one that feels push-to-talk." Streaming is what enables interruptions and quick back-and-forth turn-taking.