arXiv AI
Jun 8
dots.tts Technical Report
arXiv:2606. 07080v1 Announce Type: cross Abstract: We present dots.
Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation
arXiv:2606. 10368v1 Announce Type: cross Abstract: Speech-to-text (S2T) systems for recognition (ASR) and translation (S2TT) typically generate discrete text tokens.
Related stories
arXiv Computation and Language
Sep 15
Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation
arXiv:2609.15313v1 Announce Type: cross Abstract: Autoregressive generation of interleaved text and acoustic tokens is a common approach to spoken-response generation in speech large language models....
arXiv AI
Jul 15
Audio-Native Speech Recognition with a Frozen Discrete-Diffusion Language Model
arXiv:2607. 13013v1 Announce Type: new Abstract: Automatic speech recognition is dominated by autoregressive decoders that emit one token at a time.
Hugging Face Trending Papers
Jul 14
Audio-Native Speech Recognition with a Frozen Discrete-Diffusion Language Model
Automatic speech recognition is dominated by autoregressive decoders that emit one token at a time. We ask whether a discrete diffusion language model can transcribe speech instead, refining a whole transcript in parallel over a small number of denoising steps.
arXiv AI
Sep 2
Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models
The paper investigates whether audio‑language models capture paralinguistic cues beyond spoken content. Using the Expresso dataset and four open‑source models, the authors trace how speaking style information is encoded in the late layers of the audio encoder but is degraded before reaching the final output. They find that some models are content‑driven while others are acoustic‑driven, revealing a gap between what is encoded and what is utilized in current audio‑language models.
arXiv Machine Learning
Sep 7
GEPARD - Generative, Prosody-aware, Autoregressive text-to-speech model for Realtime Dialogue
GEPARD is a streaming text‑to‑speech model that uses a standard large language model backbone to generate speech autoregressively, decoding audio with an FSQ‑based neural codec. It streams audio chunk‑by‑chunk as text arrives, achieving a real‑time factor of about 0.067 and an aggregate speedup of roughly 204× on a single GPU with 256 concurrent streams. The design keeps all complex auxiliary mechanisms outside the decode loop, enabling deployment with a standard LLM engine (vLLM) without kernel modifications.
arXiv Computation and Language
Aug 28
FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation
FireRedAudio is a 9‑billion‑parameter audio language model that separates continuous input representations for audio understanding and speech generation, enabling a single autoregressive LLM to perform tasks such as ASR, zero‑shot TTS, Instruct TTS, and semantic/acoustic speech editing. The model uses a dedicated Audio Encoder for recognition and a RedAE‑based pathway for generation, with the LLM directly generating text or conditioning a flow‑matching DiT to produce acoustic latents. Evaluations show competitive or leading performance in multilingual ASR, content‑accurate zero‑shot TTS, strong instruction following, and significant improvements in speech editing over prior work.
arXiv AI
Jun 9
GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model
arXiv:2512. 20978v2 Announce Type: replace-cross Abstract: Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech.
arXiv AI
Aug 18
Adding Voice Cloning to Text-to-Audio-Video Models with a Single Zero-Initialised Layer
arXiv:2608. 15690v1 Announce Type: cross Abstract: Text-to-audio-video (T2AV) generation models produce a video and its soundtrack from a textual description, but offer no control over whose voice speaks in the output.
arXiv AI
Aug 11
VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference
arXiv:2608. 08569v1 Announce Type: new Abstract: Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks.
arXiv Machine Learning
Aug 17
Trajectory Dynamics in Self-Supervised Learning Latent Space for Audio Deepfake Detection
arXiv:2608. 13817v1 Announce Type: cross Abstract: Human speech production is constrained by physiology, giving rise to characteristic temporal structure on acoustic signals.
arXiv AI
Jun 9
BareWave: Waveform-Native Flow-Matching Text-to-Speech
arXiv:2606. 09048v1 Announce Type: cross Abstract: Removing intermediate representations and separately trained decoding stages has become an important direction in generative modeling.