arXiv AI
Jun 9

End-to-End Training for Discrete Token LLM based TTS System

arXiv:2606. 09234v1 Announce Type: cross Abstract: Recent state-of-the-art (SOTA) text-to-speech (TTS) systems typically adopt a cascaded pipeline consisting of a speech tokenizer, an autoregressive large language model (LLM), and a diffusion based flow-matching (FM) model, with these components trained independently.

By Changfeng Gao, Yong Ren, Jun Yuan, Ye Bai, Zhao You, ShiDong Shang
arXiv AI
Jun 9

GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model

arXiv:2512. 20978v2 Announce Type: replace-cross Abstract: Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech.

By Haoyang Li, Xuyi Zhuang, Azmat Adnan, Ye Ni, Wei Rao, Shreyas Gopal, Eng Siong Chng, Boon Siew Han, Yuanjin Zheng
arXiv AI
Aug 25

Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Text

The paper investigates how Spoken Language Models (SLMs) process speech compared to text, noting that current SLMs show weak alignment between speech and text representations despite strong downstream performance. The authors propose a framework that separates length mismatch from semantic alignment to better match speech and text representations. Experiments on multiple benchmarks demonstrate that this approach yields competitive results against strong baselines, highlighting the need to explicitly address structural differences between speech and text in SLM training.

By Hyeonyu Kim, Hwayeon Kim, Youngwon Choi, Myeongkyun Cho, Huu-Kim Nguyen
arXiv AI
Jun 10

Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation

arXiv:2606. 10368v1 Announce Type: cross Abstract: Speech-to-text (S2T) systems for recognition (ASR) and translation (S2TT) typically generate discrete text tokens.

By Xuanchen Li, Tianrui Wang, Yuheng Lu, Zikang Huang, Yu Jiang, Chenghan Lin, Chenrui Cui, Ziyang Ma, Xingyu Ma, Chunyu Qiang, Guochen Yu, Xie Chen, Longbiao Wang, Jianwu Dang