arXiv AI
D2: High-Capability Full-Duplex Spoken Language Models from Half-Duplex Backbones
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 28
Acoustic-to-Text KV Compression for Full-Duplex Speech Models
The paper introduces acoustic-to-text KV compression for full‑duplex speech models, converting acoustic key‑value states into compact textual memory during listening‑time slack. When the KV cache exceeds a target budget, older acoustic states are evicted while transcripts and recent acoustic context are retained. Experiments on ten‑minute LongSpeech sessions show a 64.6% reduction in peak streaming KV‑cache size and improved transcription, temporal question answering, and summarization, with comparable pause‑handling, turn‑taking, and interruption performance in Full‑Duplex‑Bench.
arXiv Computation and Language
Sep 18
A frontend-backend architecture for tool calls in full-duplex speech models
The paper introduces a frontend‑backend architecture for full‑duplex speech‑to‑speech models that enables tool calls while preserving natural conversational flow. The frontend emits a delegation token and streams ASR transcripts to a text‑based backend LLM, which performs tool calls and returns results that are re‑injected into the frontend via a lightweight prefill‑and‑repeat mechanism before streaming TTS synthesis. In single‑turn evaluations the system achieves 92‑97% tool‑call recall, 81.2% accuracy in rejecting irrelevant calls, and competitive performance on Full‑Duplex‑Bench‑V3 and EVA‑Bench when paired with a large backend model.
arXiv AI
Jun 9
Liberating LLM Capabilities in Full-Duplex Speech Models
arXiv:2606. 07547v1 Announce Type: cross Abstract: Speech-based large language models are typically constrained to spoken replies, which limits their user-facing outputs to what can be verbalized and suppresses text-native capabilities such as code generation, structured analysis, and multi-step reasoning in realtime interaction, for tasks that require persistent, structured, and inspectable intermediate outputs.
Hugging Face Trending Papers
Jul 7
Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs
Developing seamless, high-performance, native intelligent full-duplex Spoken Language Models (SLMs) remains a critical challenge and long-standing goal for the speech and NLP community. Despite notable progress, recent endeavors are fundamentally constrained by severe modality interference, which causes substantial knowledge degradation and compromises semantic integrity -- ultimately making full-duplex SLMs feel unnatural and unintelligent.
arXiv Computation and Language
2d ago
Diagnosing and Mitigating Long-Session Degradation in Full-Duplex Speech Models
arXiv:2610.34991v1 Announce Type: new Abstract: End-to-end full-duplex spoken dialogue models enable natural, real-time spoken interactions with interruptions, backchannels, and overlapping speech. P...
arXiv AI
Sep 21
NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities
arXiv:2609.21967v1 Announce Type: cross Abstract: We introduce NemotronLabs VoiceChat, an open full-duplex speech-to-speech model with native tool-calling capabilities. NemotronLabs VoiceChat combine...