The study evaluates conversational speech models by placing them at 427 pauses and 482 turn ends from 176 unscripted conversations, scoring their responses on timing, latency, length, wording, and recall of recent dialogue. Hosted APIs (GPT Realtime, Gemini 3.1 Flash Live, Grok Voice) respond at almost every pause, with latency ranging from 1–4 s for silence-based endpointers and about 8 s for semantic turn detection, while full‑duplex models (PersonaPlex, Moshi) remain silent more often and can interrupt within human reaction time. Recall rates vary: hosted systems remember 41–60 % of facts under a Claude Opus 5.5 judge and 19–26 % with a strict word matcher; full‑duplex models recall 17–22 % and 12–15 % respectively.
By Arya Bastani, Iman Modarressi
arXiv:2609.13445v1 Announce Type: new
Abstract: Speech-to-speech LLMs like Moshi, and its derivative PersonaPlex, can listen and speak concurrently through full-duplex generation. However, they can b...
By Kento Nishi
arXiv:2606. 07547v1 Announce Type: cross Abstract: Speech-based large language models are typically constrained to spoken replies, which limits their user-facing outputs to what can be verbalized and suppresses text-native capabilities such as code generation, structured analysis, and multi-step reasoning in realtime interaction, for tasks that require persistent, structured, and inspectable intermediate outputs.
By Luoyuan Zhang, Bokai Xu, Junbo Cui, Weiyue Sun, Yingjing Xu, Hanyu Liu, Yuan Yao
arXiv:2610.08683v1 Announce Type: new
Abstract: Full-duplex speech models are trained to converse with a person, but they are increasingly made to converse with each other, in self-play data generati...
By Lichen Zhu, Yueqian Lin, Yiheng Wang, Hai "Helen" Li, Yiran Chen
DiffuPlex is a rolling masked diffusion framework designed to accelerate full‑duplex spoken dialog models by predicting multiple future user and assistant frames in a single backbone pass. It consumes only a confident prefix of each predicted future while maintaining the original frame rate, and revises unplayed future content when the interaction diverges. Two inference policies—DiffuPlex‑LISTEN and DiffuPlex‑SPEAK—offer speedups of 1.46×–1.59× in deployment‑path wall‑clock time and 1.61×–1.80× in Core LM speed, with all backbone invocations completing within the 80 ms interaction interval, while preserving conversational quality and, in the LISTEN policy, speech naturalness.
By Heeseung Kim
arXiv:2610.34991v1 Announce Type: new
Abstract: End-to-end full-duplex spoken dialogue models enable natural, real-time spoken interactions with interruptions, backchannels, and overlapping speech. P...
By Junho Kim, CheHyun Lee, Donghun Kang, Youngho Cho, Dongwook Lee, Heeseung Kim, Sungroh Yoon
arXiv:2606. 11386v1 Announce Type: cross Abstract: Full-duplex spoken language models (FD-SLMs) enable seamless speech interaction by allowing models to listen and speak simultaneously, yet the internal mechanism by which they coordinate listening and speaking remains underexplored.
By Cheng-Kuang Chang, Kai-Wei Chang, Alexander H. Liu, James Glass
Full-duplex speech models require training data that preserves turn-taking, overlap, interruption, and backchannel behavior, yet these signals are entangled across speakers in noisy real-world recordi...
arXiv:2610.35581v1 Announce Type: cross
Abstract: A full-duplex spoken dialogue model must yield when the user cuts in and hold when the user merely backchannels. Sequence-level RL with a timing rewa...
By Guoqiang Hu, Lichao Zhang, Yizhou Peng, Yuhao Luo, Haifeng Hu, Sallen Sun, Will Yang, Eng Siong Chng
DuplexLedger is a bilingual benchmark comprising 1,094 persistent sessions that align four full‑duplex scenarios with event‑level Write/Update/Ignore/Retain supervision, explicit pre/post memory states, and 6,564 same‑session questions and checks. Evaluation of five full‑duplex models shows an interaction‑memory gap, with no system achieving strong boundary control and coherent final‑state memory; the best final‑state fact coverage is only 2.01%. The authors propose ACT‑Mem, an 8.92M‑parameter sidecar that improves latest‑value accuracy from 4.11% to 8.68% on held‑out sessions while reducing stale assertions and distractor adoption, though it slows interruption stopping.
By Yuxuan Li, Xiangchen Guan, Kunrong Li, Jin Wang, Kaiwen Luo, Yang Xiao, Yufei Guo, Zhisheng Chen, Liang Lin, Kun Wang
arXiv:2510. 05150v3 Announce Type: replace-cross Abstract: Recent advances in spoken dialogue language models (SDLMs) reflect growing interest in shifting from turn-based to full-duplex systems, where the models continuously perceive user speech streams while generating responses.
By Donghang Wu, Haoyang Zhang, Chen Chen, Tianyu Zhang, Fei Tian, Xuerui Yang, Gang Yu, Hexin Liu, Nana Hou, Yuchen Hu, Eng Siong Chng
A full-duplex spoken dialogue model must yield when the user cuts in and hold when the user merely backchannels. Sequence-level RL with a timing reward (GRPO) is the usual recipe; we test its whole-ro...