← Back to all news
arXiv AI By Yingzhuo Deng, Chong Li, Bingxin Liu, Yi Shu, Tianyu Peng, Can Xie, Siyao Song, Shuo Ren, Jiajun Zhang

CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models

Read the original on arXiv AI →

CoSTA is a post‑hoc framework that decouples the generation and synthesis token spaces in Spoken Dialogue Models, allowing the use of compressed speech tokens for faster generation while preserving the original tokenizer and decoder. It extends the generation token space with compressed representations through mean‑initialized vocabulary extension, pronunciation adaptation, and dialogue adaptation, then deterministically expands back to the original token space for synthesis. Experiments on bilingual speech‑dialogue benchmarks show about 1.7× faster generation, lower latency, higher throughput, and maintained response quality, with a practical trade‑off between compression, decoding efficiency, and quality.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Oct 5

Learning When to Commit from Partial Speech for End-to-End Simultaneous Speech Translation

The paper presents a method for improving simultaneous speech translation by adapting a full‑utterance speech language model with prefix supervision derived from its own complete and partial waveform translations, eliminating the need for transcripts or human translations. Experiments on FLEURS and CoVoST2 across three language directions show that prefix training enhances quality–latency trade‑offs, especially when combined with multi‑turn append‑only decoding, and that a confidence threshold effectively controls the inference‑time quality–latency balance. The study also explores the impact of synthesis margin on translation quality and calibration, finding a non‑monotonic relationship with latency.

By Hieu Hoang, Amittai Axelrod
llms
More like this →
Hugging Face Trending Papers
Jun 2

Efficient ASR Training with Conversations that Never Happened

Conversational ASR for lower-resource languages and niche domains is limited by the scarcity of domain-matched multi-speaker training data. We propose an augmentation pipeline that generates scenario-level dialogues with participant metadata, maps speaker attributes to TTS voice profiles, and assembles synthesized utterances into speaker-aware simulated conversations.

llmsmultimodalbenchmarks
More like this →