arXiv AI
Jul 9
Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
arXiv:2607. 06831v1 Announce Type: cross Abstract: Speech-to-text alignment means finding the temporal boundaries of each word in the audio.
End-to-End Training for Discrete Token LLM based TTS System
arXiv:2606. 09234v1 Announce Type: cross Abstract: Recent state-of-the-art (SOTA) text-to-speech (TTS) systems typically adopt a cascaded pipeline consisting of a speech tokenizer, an autoregressive large language model (LLM), and a diffusion based flow-matching (FM) model, with these components trained independently.
Related stories
arXiv AI
Jun 8
dots.tts Technical Report
arXiv:2606. 07080v1 Announce Type: cross Abstract: We present dots.
arXiv AI
Aug 5
GROW: Group-Relative Advantage-Weighted On-Policy Reinforcement Learning of Autoregressive-Diffusion Text-to-Speech model
arXiv:2608. 03215v1 Announce Type: cross Abstract: Reinforcement learning for flow-matching text-to-speech is complicated by deterministic ODE sampling: trajectory-level policy-gradient methods typically convert the ODE into an SDE and track per-step likelihood ratios, introducing stochastic perturbations and substantial overhead.
arXiv AI
Aug 11
Length-MAX Tokenizer for Language Models
arXiv:2511. 20849v2 Announce Type: replace-cross Abstract: We introduce a new tokenizer for language models that minimizes the average tokens per character, thereby reducing the number of tokens needed to represent text during training and to generate text during inference.
arXiv AI
Jun 16
LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization
arXiv:2506. 16738v2 Announce Type: replace-cross Abstract: With the rapid progress of speech language models (SLMs), discrete speech tokens have emerged as a core interface between speech and text, enabling unified modeling across modalities.
arXiv AI
Jun 9
GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model
arXiv:2512. 20978v2 Announce Type: replace-cross Abstract: Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech.
arXiv AI
Jun 6
Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning
arXiv:2606. 05173v1 Announce Type: cross Abstract: Masked language modelling (MLM) has been the dominant pre-training objective for text encoders since BERT, yet it encourages representations that are strongly anchored to surface-form token identity rather than deeper semantic structure.
arXiv AI
Jul 7
Unified Audio Intelligence Without Regressing on Text Intelligence
arXiv:2607. 05196v1 Announce Type: cross Abstract: Audio intelligence involves understanding, reasoning about, and generating both audio and speech.
arXiv AI
Jul 1
CHERRY: Compressed Hierarchical Experts with Recurrent Representational Yield
arXiv:2606. 31796v1 Announce Type: cross Abstract: We study three complementary techniques for training compute-efficient language models.
arXiv AI
Jul 7
TokAN: Accent Normalization Using Self-Supervised Speech Tokens
arXiv:2607. 03928v1 Announce Type: cross Abstract: Accent normalization (AN) seeks to convert non-native (L2) accented speech into standard (L1) speech while preserving speaker identity.
Hugging Face Trending Papers
Aug 12
Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder
Recent advances in zero-shot text-to-speech (TTS) have substantially improved speech quality and voice cloning fidelity. However, many zero-shot TTS systems still depend on audio prompt transcripts at inference time.
arXiv AI
Jun 16
WavSLM: Single-Stream Speech Language Modeling via WavLM Distillation
arXiv:2603. 05299v2 Announce Type: replace-cross Abstract: Large language models show that simple autoregressive training can yield scalable and coherent generation, but extending this paradigm to speech remains challenging due to the entanglement of semantic and acoustic information.