arXiv:2610.27109v1 Announce Type: cross Abstract: Spoken interaction is moving from alternating turns to full-duplex conversation, where an agent listens while speaking. Learning this coordination re...
arXiv:2606.11167v2 Announce Type: replace Abstract: Full-duplex spoken dialogue models can listen and speak simultaneously, making them a promising architecture for natural conversation. However, cur...
arXiv:2609.13117v1 Announce Type: new Abstract: Full-duplex evaluation often emphasizes whether an agent keeps speaking or stops. That binary cannot express a third response humans use routinely: con...
The paper introduces a pipeline that generates intent‑labeled, two‑channel conversational speech from relational event lists, enabling controlled synthesis of full‑duplex dialogue with 42 phenomena across eight families in English and Mandarin. By having an LLM author each event’s speaker, text, conversational act, and attachment, and then aligning and timing these events independently, the system produces diverse, realistic turn‑taking signals. Experiments show that models trained on this synthetic corpus achieve higher floor‑occupancy accuracy and better start‑speaking/listening F1 scores compared to models trained on prior data.
arXiv:2610.34991v1 Announce Type: new Abstract: End-to-end full-duplex spoken dialogue models enable natural, real-time spoken interactions with interruptions, backchannels, and overlapping speech. P...
The paper introduces an automated pipeline that extracts conversational turns and backchannels from separate-channel recordings of spontaneous dyadic dialogue, combining voice activity detection, channel-energy filtering, temporal merging, automatic speech recognition, and context-based post‑processing. Evaluated on 99 ten‑minute Danish conversations, the system achieved F1 scores around 0.62 for both turns and backchannels, with median onset/offset errors of roughly 0.15–0.18 s. Performance was consistent across normal and asymmetric listening conditions, and a case study showed the pipeline’s outputs were less variable than human annotations, supporting its use as a reliable first‑pass annotation tool in semi‑automated workflows.
arXiv:2606. 07547v1 Announce Type: cross Abstract: Speech-based large language models are typically constrained to spoken replies, which limits their user-facing outputs to what can be verbalized and suppresses text-native capabilities such as code generation, structured analysis, and multi-step reasoning in realtime interaction, for tasks that require persistent, structured, and inspectable intermediate outputs.
SteerDuplex is a full‑duplex speech dialogue model that can be steered along attributes such as tone, persona, speaking rate, and voice style in response to user instructions. The authors introduce a taxonomy of text‑ and audio‑based steerability, identify gaps in existing models, and fine‑tune a Moshi‑based model with reinforcement learning to improve timing and response continuity. They also present SteerBench, a benchmark of 390 spoken prompts and 1,067 human‑authored rubrics, showing significant gains in audio‑steering pass rates and interruption handling compared to open baselines.
DuplexDrama is a newly announced synthesized spoken dialogue dataset that uniquely combines complete persona and scenario settings, three full‑duplex behaviors (interruption, backchannel, incomplete), expressive speech with persona‑aligned emotion labels, and script‑aware sound events. The dataset was created through a four‑stage pipeline and validated for quality on both scripts and audio, yielding over 2,000 hours of audio featuring 64 voices across 13 personas and 5 age groups, with 3.8% of turns containing full‑duplex behaviors. A curated bilingual subset of 6,400 dialogues (800 hours total) will be released to support research in full‑duplex spoken dialogue models, and evaluation prompts will accompany the dataset.
arXiv:2610.20172v1 Announce Type: cross Abstract: Full-duplex speech systems must decide when to enter, hold, or relinquish the conversational floor, even as multiple speakers overlap or backchannel...
The paper introduces Alignment-Free Text‑Audiobox (Text‑AB), a unified diffusion‑based framework that performs high‑quality voice dubbing and full‑duplex dialogue synthesis without requiring forced alignment. Text‑AB uses a latent diffusion model with DAC‑VAE features, achieving over 10× compression compared to prior EnCodec representations, and learns text‑speech alignment via cross‑attention. The authors pretrain a 3B‑parameter model on 480k hours of monolingual speech and fine‑tune it for cross‑lingual dubbing, full‑duplex dialogue, and emotional dialogue, reporting significant improvements in prosody, voice similarity, naturalness, and emotional expressivity over existing internal systems.
arXiv:2610.32349v1 Announce Type: new Abstract: Full-duplex speech models listen and speak at the same time, which makes them promising for natural, open-ended conversation. But can they sustain a co...