People Making Conversation: Evaluating How Naturally Speech Models Converse

Key points

• Across six systems, the hosted APIs (GPT Realtime, Gemini 3.1 Flash Live, Grok Voice) respond at almost every pause whichever endpointer they use; only their latency, about 1 to 4 s with silence-based endpointers and about 8 s with semantic turn detection, decides whether the words land inside the pause.1
• The full-duplex models (PersonaPlex, Moshi) are the only systems that regularly stay silent and the only ones that interrupt within a human reaction time.1
• Hosted systems reply longer than the person.1
• Given the history as text, hosted systems recall 41 to 60% of conversation facts under a Claude Opus 5.5 judge and 19 to 26% under a strict word matcher; hearing it as audio, full-duplex models recall 17 to 22% and 12 to 15%.1

Sources (1)

• [1]People Making Conversation: Evaluating How Naturally Speech Models Converse
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 11:18 PM

  A conversational speech model has to decide, at every pause, whether to speak.

  Across six systems, the hosted APIs (GPT Realtime, Gemini 3.1 Flash Live, Grok Voice) respond at almost every pause whichever endpointer they use; only their latency, about 1 to 4 s with silence-based endpointers and about 8 s with semantic turn detection, decides whether the words land inside the pause.

Extractive summary: sentences quoted from the sources.