arXiv AI
Coupled but Late: Turn-Taking Between Full-Duplex Speech Models in Unscripted Dialogue
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 25
Controlling Backchannels in Streamable Full-duplex Models
The paper introduces a lightweight backchannel head that predicts when a backchannel should begin in full-duplex spoken dialogue models, using the models’ hidden states. When the predicted probability exceeds a tunable threshold, a backchannel is force‑decoded. Experiments on 7B and 1B models show that the head generalizes across scale, aligns with human timing, and produces backchannels that human raters judge as comparable to real ones.
arXiv AI
3d ago
Reference-Guided Turn-Taking Style Transfer for Full-Duplex Spoken Language Models
The paper introduces a method for controlling the turn‑taking behavior of full‑duplex spoken language models by identifying a single late‑layer residual‑stream direction, called the initiative vector, that determines how much the model speaks. By extracting this vector from the model’s own conversations without additional training, the authors can anticipate speech onsets up to 800 ms ahead and adjust the speech ratio from 6–12 % to 42–61 % through scaling during generation. A short reference recording of a target speaker allows the model to adopt that speaker’s turn‑taking style, and the transfer is highly speaker‑specific, with mismatched references degrading performance in 93–98 % of cases.
arXiv Computation and Language
4d ago
MultiPartyTurnBench: Benchmarking Floor Control in Multi-Party Spoken Dialogue
arXiv:2610.20172v1 Announce Type: cross Abstract: Full-duplex speech systems must decide when to enter, hold, or relinquish the conversational floor, even as multiple speakers overlap or backchannel...
arXiv Computation and Language
Sep 14
Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agents
arXiv:2609.13117v1 Announce Type: new Abstract: Full-duplex evaluation often emphasizes whether an agent keeps speaking or stops. That binary cannot express a third response humans use routinely: con...
arXiv Computation and Language
Oct 7
Conversation Is a Two-Body Problem: Dyadic Evaluation of Full-Duplex Dialogue Models
arXiv:2610.08125v1 Announce Type: cross Abstract: Full-duplex spoken dialogue models listen and speak at the same time, enabling voice agents to have natural, low-latency interactions that turn-based...
arXiv Computation and Language
Oct 12
People Making Conversation: Evaluating How Naturally Speech Models Converse
The study evaluates conversational speech models by placing them at 427 pauses and 482 turn ends from 176 unscripted conversations, scoring their responses on timing, latency, length, wording, and recall of recent dialogue. Hosted APIs (GPT Realtime, Gemini 3.1 Flash Live, Grok Voice) respond at almost every pause, with latency ranging from 1–4 s for silence-based endpointers and about 8 s for semantic turn detection, while full‑duplex models (PersonaPlex, Moshi) remain silent more often and can interrupt within human reaction time. Recall rates vary: hosted systems remember 41–60 % of facts under a Claude Opus 5.5 judge and 19–26 % with a strict word matcher; full‑duplex models recall 17–22 % and 12–15 % respectively.