arXiv AI September 28, 2026 By Yejin Lee, Seungbeom Kim, Yongha Lee, Kyuhong Shim

Acoustic-to-Text KV Compression for Full-Duplex Speech Models

The paper introduces acoustic-to-text KV compression for full‑duplex speech models, converting acoustic key‑value states into compact textual memory during listening‑time slack. When the KV cache exceeds a target budget, older acoustic states are evicted while transcripts and recent acoustic context are retained. Experiments on ten‑minute LongSpeech sessions show a 64.6% reduction in peak streaming KV‑cache size and improved transcription, temporal question answering, and summarization, with comparable pause‑handling, turn‑taking, and interruption performance in Full‑Duplex‑Bench.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 11

VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

arXiv:2608. 08569v1 Announce Type: new Abstract: Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks.

By Wenxu Jia, Dongjie Fu, Xize Cheng, Fangming Feng, Linjun Li, Wenshi Chen, Yingming Li, Zhou Zhao, Tao Jin

llms efficiency multimodal benchmarks safety

arXiv AI

Jun 9

Liberating LLM Capabilities in Full-Duplex Speech Models

arXiv:2606. 07547v1 Announce Type: cross Abstract: Speech-based large language models are typically constrained to spoken replies, which limits their user-facing outputs to what can be verbalized and suppresses text-native capabilities such as code generation, structured analysis, and multi-step reasoning in realtime interaction, for tasks that require persistent, structured, and inspectable intermediate outputs.

By Luoyuan Zhang, Bokai Xu, Junbo Cui, Weiyue Sun, Yingjing Xu, Hanyu Liu, Yuan Yao

llms data-engineering

arXiv Machine Learning

Jul 23

Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting

arXiv:2607. 20086v1 Announce Type: cross Abstract: State-space sequence models are attractive for streaming speech because they maintain compact recurrent state, but scan-style training kernels can have unfavorable constants for short audio tasks.

By Mahesh Godavarti

benchmarks

arXiv Computation and Language

Sep 15

Enabling Streaming User Transcription in Full-Duplex Speech-to-Speech Models

The paper introduces a lightweight ASR head that can be added to full‑duplex speech‑to‑speech models, enabling real‑time user transcription without major architectural changes. The method adds only a few parameters and preserves full‑duplex conversational features such as turn‑taking and barge‑in. Experiments show a streaming WER of 10.21% within the duplex framework and 7.73% when trained as a standalone ASR model, matching state‑of‑the‑art performance.

By Ke Hu, Nourchene Ferchichi, Edresson Casanova, Ankita Pasad, Elena Rastorgueva, Chen Chen, Nithin Rao Koluguri, Piotr Zelasko, Yifan Peng, Hainan Xu, Zhehuai Chen, Boris Ginsburg

agents multimodal benchmarks

Hugging Face Trending Papers

Sep 8

ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansion

Full-duplex speech models require training data that preserves turn-taking, overlap, interruption, and backchannel behavior, yet these signals are entangled across speakers in noisy real-world recordi...

llms safety

arXiv Computation and Language

Aug 25

WnW: Waxing-and-Waning KV Cache for Long-Form Speech LLMs

arXiv:2608.22704v1 Announce Type: new Abstract: Long-form audio inputs make the KV cache the dominant memory cost of speech LLMs. Prefill-only KV compression methods permanently discard audio KV posi...

By Yiming Yao, Chenyang Lyu, Xuanfan Ni, Longyue Wang, Weihua Luo, Yazheng Yang, Jinsong Su

llms efficiency