The paper introduces a lightweight backchannel head that predicts when a backchannel should begin in full-duplex spoken dialogue models, using the models’ hidden states. When the predicted probability exceeds a tunable threshold, a backchannel is force‑decoded. Experiments on 7B and 1B models show that the head generalizes across scale, aligns with human timing, and produces backchannels that human raters judge as comparable to real ones.
By Maike Z\"ufle, Peter Pol\'ak, Sefik Emre Eskimez, Jan Niehues, Peter Bell, Ond\v{r}ej Klejch
The paper introduces a method for controlling the turn‑taking behavior of full‑duplex spoken language models by identifying a single late‑layer residual‑stream direction, called the initiative vector, that determines how much the model speaks. By extracting this vector from the model’s own conversations without additional training, the authors can anticipate speech onsets up to 800 ms ahead and adjust the speech ratio from 6–12 % to 42–61 % through scaling during generation. A short reference recording of a target speaker allows the model to adopt that speaker’s turn‑taking style, and the transfer is highly speaker‑specific, with mismatched references degrading performance in 93–98 % of cases.
By Yuxuan Chen, Yang Xiao, Jack Sun, Chak Sun, Brick Li, Sam
arXiv:2610.20172v1 Announce Type: cross
Abstract: Full-duplex speech systems must decide when to enter, hold, or relinquish the conversational floor, even as multiple speakers overlap or backchannel...
By Puneet Mathur, Dinesh Manocha
arXiv:2609.13117v1 Announce Type: new
Abstract: Full-duplex evaluation often emphasizes whether an agent keeps speaking or stops. That binary cannot express a third response humans use routinely: con...
By Yunqi Lu, Tyler Baumgartner, Nikhil Johri, Brandon Tai, Candice Fan, Luc Debaupte, Ruben Aguilar, Bill Wang, Yi Zhong
arXiv:2610.08125v1 Announce Type: cross
Abstract: Full-duplex spoken dialogue models listen and speak at the same time, enabling voice agents to have natural, low-latency interactions that turn-based...
By Sungnyun Kim, Sungwoo Cho, Jihwan Oh, Se-Young Yun
The study evaluates conversational speech models by placing them at 427 pauses and 482 turn ends from 176 unscripted conversations, scoring their responses on timing, latency, length, wording, and recall of recent dialogue. Hosted APIs (GPT Realtime, Gemini 3.1 Flash Live, Grok Voice) respond at almost every pause, with latency ranging from 1–4 s for silence-based endpointers and about 8 s for semantic turn detection, while full‑duplex models (PersonaPlex, Moshi) remain silent more often and can interrupt within human reaction time. Recall rates vary: hosted systems remember 41–60 % of facts under a Claude Opus 5.5 judge and 19–26 % with a strict word matcher; full‑duplex models recall 17–22 % and 12–15 % respectively.
By Arya Bastani, Iman Modarressi
arXiv:2609.39446v1 Announce Type: new
Abstract: Existing full-duplex speech benchmarks cover only subsets of real-time interaction behaviors, often under limited contextual conditions. We introduce D...
By Keyue Xing, Wentao Ding, Mengmeng Wang, Wenming Tu, Zilong Zheng, Yipeng Kang
arXiv:2609.27372v1 Announce Type: cross
Abstract: Benchmarks for full-duplex spoken dialogue models score turn-taking with binary fixed-window rules that reward immediate response or silence by compl...
By Kian Shamsaie, Iman Modarressi
ECHO is a new paired diagnostic benchmark designed for Chinese full‑duplex spoken dialogue systems to evaluate context‑sensitive turn‑taking. It pairs examples that share the same overlap transcript but differ in preceding multi‑turn context, requiring either a Yield or Keep decision, and also includes off‑talk cases to test unnecessary yielding. The benchmark introduces pair accuracy, penalizing constant‑action policies, and shows that many systems are biased toward Yield, performing better on interruptions than on backchannels.
By Shuofeng Zhao, Hongwei Cai, Wenke Fan, Qingxiang Guo, Dawei Yang, Zhou Wang, Zhiyang Zhou, Yingxin Shang, Weixu Wang, Lin Yang, Shuran Zhou, Yang Song
arXiv:2608.28630v1 Announce Type: cross
Abstract: Compared with half-duplex dialogue systems where the system waits for user turn completion before it responds, natural full-duplex dialogue systems r...
By Tianrui Pan, Qinglin Zhang, Chong Deng, Luyao Cheng, Qian Chen, Wen Wang, Jie Tang, Gangshan Wu, Jie Liu
TurnBench is a new multi‑domain benchmark for evaluating turn‑taking dynamics in spoken dialogue. It comprises a 30‑hour hand‑labeled corpus of dyadic human conversations, a standardized evaluation protocol for end‑of‑turn and interruption detection, and covers six distinct interaction styles with triple annotation. The benchmark also provides a 104‑hour training set, a public leaderboard, and an interactive dataset viewer at https://turnbench.sesame.com.
By Freeman Jiang, Ramon Sanabria, Soham Deshmukh, Bandhav Veluri, Simon Michael Vuch Williams, Elliott K. Suen, Garreth Lee, Kevin Yoonho Choi, Takuya Umeki, Riku Kubo, Sathvik Udupa, Chien-yu Huang, Shih-Yun Shan Kuan, Zhuoyan Tao, Satyapriya Krishna, Sefik Emre Eskimez, Yu Tsao, Hung-yi Lee, Shinji Watanabe
Full-duplex speech models require training data that preserves turn-taking, overlap, interruption, and backchannel behavior, yet these signals are entangled across speakers in noisy real-world recordi...