arXiv Machine Learning
Aug 7
LILAC: An Idempotent Neural Speech Codec
arXiv:2608. 05727v1 Announce Type: cross Abstract: Neural Audio Codecs are widely adopted in speech generation and editing.
CleanCodec: Efficient and Robust Speech Tokenization via Perceptually Guided Encoding
Neural audio codecs are a key component of speech processing pipelines, compressing audio into discrete tokens for downstream modeling. However, existing codecs struggle to balance reconstruction quality with token efficiency, often encoding perceptually irrelevant information such as background noise and recording artifacts at the expense of linguistically and acoustically meaningful content.
Related stories
arXiv AI
Jun 3
EntangleCodec: A Unified Discrete Audio Tokenizer via Semantic-Acoustic Entanglement
arXiv:2606. 02739v1 Announce Type: cross Abstract: Audio tokenizers serve as the discrete interface between continuous audio and Audio Language Models (ALMs), but existing tokenizers often struggle to support both understanding and generation.
Hugging Face Trending Papers
Jul 6
Unified Audio Intelligence Without Regressing on Text Intelligence
Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM.
arXiv AI
Jun 2
DSA-Tokenizer: Disentangled Semantic-Acoustic Tokenization via Flow Matching-based Hierarchical Fusion
arXiv:2601. 09239v5 Announce Type: replace-cross Abstract: Speech tokenizers are a key building block of fully discrete Speech LLMs.
arXiv AI
Jun 10
Whisper-GPT -- Continuous Discrete Hybrid Representation Language Models For Speech And Music
arXiv:2412. 11449v2 Announce Type: replace-cross Abstract: We propose WHISPER-GPT: A generative large language model (LLM) for speech and music that allows us to work with continuous audio representations and discrete tokens simultaneously as part of a single architecture.
arXiv AI
Jun 8
HybridCodec: Fast Dual-Stream, Semantically Enhanced Neural Audio Codec
arXiv:2606. 06743v1 Announce Type: cross Abstract: The popularity of neural audio codecs as speech tokenizers has surged with the advent of Multimodal Large Language Models.
Hugging Face Trending Papers
Jun 10
Feature-Aligned Speech Watermarking for Robustness to Reconstruction Distortions
Audio watermarking aims to embed identifiable information into audio while remaining imperceptible. Existing methods adopt high-fidelity, low-energy designs to preserve perceptual quality, but the resulting watermarks lack robustness under suppression by speech reconstruction models.
arXiv AI
Jun 6
F3-Tokenizer: Taming Audio Autoencoder Latents for Understanding and Generation
arXiv:2606. 06357v1 Announce Type: cross Abstract: Continuous audio autoencoders reconstruct waveforms well but often produce latents with weak structure for understanding, while self-supervised audio encoders capture semantics but are not directly decodable.
arXiv AI
Jun 11
Feature-Aligned Speech Watermarking for Robustness to Reconstruction Distortions
arXiv:2606. 11828v1 Announce Type: cross Abstract: Audio watermarking aims to embed identifiable information into audio while remaining imperceptible.
arXiv AI
Jun 8
Benchmarking Language Modeling for Lossless Compression of Full-Fidelity Audio
arXiv:2603. 08683v2 Announce Type: replace-cross Abstract: Autoregressive "language" models (LMs) trained on raw waveforms can be repurposed for lossless audio compression, but prior work is limited to 8-bit audio, leaving open whether such approaches work for practical settings (16/24-bit) and can compete with existing codecs.
arXiv AI
Jul 7
Unified Audio Intelligence Without Regressing on Text Intelligence
arXiv:2607. 05196v1 Announce Type: cross Abstract: Audio intelligence involves understanding, reasoning about, and generating both audio and speech.
arXiv AI
Jun 2
HoliTok:A Coutinuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding
arXiv:2605. 29948v2 Announce Type: replace-cross Abstract: Unified speech foundation models require a holistic tokenization space that is both learnable by language models and decodable into high-quality waveforms.