arXiv AI June 9, 2026 By Changfeng Gao, Yong Ren, Jun Yuan, Ye Bai, Zhao You, ShiDong Shang

End-to-End Training for Discrete Token LLM based TTS System

arXiv:2606. 09234v1 Announce Type: cross Abstract: Recent state-of-the-art (SOTA) text-to-speech (TTS) systems typically adopt a cascaded pipeline consisting of a speech tokenizer, an autoregressive large language model (LLM), and a diffusion based flow-matching (FM) model, with these components trained independently.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Sep 15

Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation

arXiv:2609.15313v1 Announce Type: cross Abstract: Autoregressive generation of interleaved text and acoustic tokens is a common approach to spoken-response generation in speech large language models....

By Daxin Tan, Dehua Tao, Chengxi Deng, Hanlin Zhang, Xiao Chen

llms fine-tuning efficiency multimodal

arXiv Computation and Language

5d ago

Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoring

The paper introduces a training‑free speech‑and‑text‑to‑pronunciation (ST2P) pipeline that combines lexical candidates from G2P tools with acoustic rescoring using frozen pretrained S2P models. By performing a left‑to‑right greedy search over whole‑sequence negative log‑likelihoods, the method achieves a dramatic reduction in character error rate on Japanese corpora, outperforming both baseline G2P/S2P approaches and commercial multimodal LLMs. The approach is also significantly faster—3–3.5× faster than beam search and twice as fast as direct decoding—while maintaining high accuracy across multiple languages.

By Hikaru Asano, Yotaro Kubo, So Kuroki

llms multimodal

arXiv AI

Jul 9

Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs

arXiv:2607. 06831v1 Announce Type: cross Abstract: Speech-to-text alignment means finding the temporal boundaries of each word in the audio.

By Albert Zeyer, Ralf Schl\"uter, Hermann Ney

llms multimodal safety

arXiv Computation and Language

Sep 1

A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification

arXiv:2512.07571v3 Announce Type: replace Abstract: This paper presents a simple method that allows to easily enhance textual pre-trained large language models with speech information, when fine-tune...

By Nicolas Calbucura, Jose Guillen, Valentin Barriere

llms rag nlp fine-tuning multimodal

arXiv AI

Jun 8

dots.tts Technical Report

arXiv:2606. 07080v1 Announce Type: cross Abstract: We present dots.

By Shi Lian, Changtao Li, Bohan Li, Hankun Wang, Da Zheng, Junfeng Tian, Yufeng Ma, Colin Zhang, Kai Yu

diffusion efficiency multimodal benchmarks

arXiv AI

Sep 15

Bridging the Modality Gap in Long-Form Clinical Audio: A Comparative Study of Lightweight and Heavyweight End-to-End SOAP Generation

arXiv:2609.14467v1 Announce Type: cross Abstract: Automating clinical documentation from long-form doctor-patient conversations remains challenging for modern audio-language models. While cascaded AS...

By Ziyu Zhang, Mingchen Shao, Wenjie Tian, Tianlun Zuo, Longhao Li, Lei Xie

llms nlp fine-tuning multimodal safety

One email a morning, machine-written

Related stories

Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation

Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoring

Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs

A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification

dots.tts Technical Report

Bridging the Modality Gap in Long-Form Clinical Audio: A Comparative Study of Lightweight and Heavyweight End-to-End SOAP Generation