arXiv AI

Shao: Scaling Acoustic Token Language Models Toward High-Fidelity Music Generation

arXiv:2605. 01790v2 Announce Type: replace-cross Abstract: A common design pattern in high-quality music generation is to handle structure and fidelity in different representation spaces: a generator first models high-level structure, followed by diffusion-based or neural decoding stages that reconstruct fine details.

Hugging Face Trending Papers

Jul 22

Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering

In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes. The proposed framework supports three tasks: Lyrics-to-Song Generation, which generates complete songs from text descriptions, lyrics, and musical attributes; Instrumental Music Generation, which creates music without vocals; and Cover Song Generation, which reinterprets existing songs with different styles while preserving their melodic content.

diffusion nlp reinforcement-learning benchmarks

arXiv AI

Jul 23

Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering

arXiv:2607. 20253v1 Announce Type: cross Abstract: In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes.

By Junyu Dai, Xinyue Fan, Weiqin Li, Xiangang Li, Yunjia Li, Bin Ma, Yukun Ma, Chongjia Ni, Yufei Shi, Haoxu Wang, Menglin Wu, Jianwei Yu, Huaicheng Zhang, Han Zhao, Shengkui Zhao, Haina Zhu

diffusion nlp reinforcement-learning benchmarks

arXiv AI

Jun 2

HoliTok:A Coutinuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding

arXiv:2605. 29948v2 Announce Type: replace-cross Abstract: Unified speech foundation models require a holistic tokenization space that is both learnable by language models and decodable into high-quality waveforms.

By Bohan Li, Shi Lian, Hankun Wang, Yiwei Guo, Yu Xi, Zhihan Li, Da Zheng, Colin Zhang, Kai Yu

llms nlp

arXiv Machine Learning

Jun 8

Making the Most of Limited Data: Score-Aware Training for Text-to-Music Generation

arXiv:2606. 07387v1 Announce Type: new Abstract: State-of-the-art text-to-music generation systems rely on massive proprietary datasets and industrial-scale compute, making it impossible to disentangle architectural contributions from resource advantages.

By Yun-Chen Cheng, Tzu-Hung Huang, Chih-Pin Tan

benchmarks safety

arXiv AI

Jul 7

Unified Audio Intelligence Without Regressing on Text Intelligence

arXiv:2607. 05196v1 Announce Type: cross Abstract: Audio intelligence involves understanding, reasoning about, and generating both audio and speech.

By Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim, Boxin Wang, Zihan Liu, Sungwon Kim, Yang Chen, Arushi Goel, Rajarshi Roy, Wenliang Dai, Zhuolin Yang, Yangyi Chen, Dongfu Jiang, Sreyan Ghosh, Tuomas Rintamaki, Andrew Tao, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping

llms rag agents reinforcement-learning efficiency multimodal benchmarks safety

arXiv AI

Aug 3

Stable Autoregressive Speech Generation with Low-Frame-Rate High-Dimensional Continuous Tokens

arXiv:2607. 29363v1 Announce Type: cross Abstract: Balancing sequence length, representational capacity, and long-horizon stability is a central problem in autoregressive (AR) speech and audio generation.

By Yi Luo, Rongzhi Gu, Jixun Yao

llms multimodal

arXiv AI

Jun 2

JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions

arXiv:2606. 01703v1 Announce Type: cross Abstract: We address the challenge of generating high-fidelity, long-form soundtracks that remain coherent across scene transitions.

By Jiashuo Yu, Yao Yao, Boyu Chen, Alex Wang

llms agents multimodal benchmarks safety

arXiv Machine Learning

Jun 3

SketchSong: Hierarchical Song Generation with Sketch Planning and Fine-Grained Multi-Track Modeling

arXiv:2606. 03169v1 Announce Type: cross Abstract: Recent song generation systems can synthesize realistic audio, yet generating complete songs remains challenging for two reasons.

By Xiaoyue Duan, Nanxing Hu, Yutang Feng, Xudong Yan, Jiatao Chen, Jinchao Zhang, Jie Zhou

benchmarks

Hugging Face Trending Papers

Jul 6

Unified Audio Intelligence Without Regressing on Text Intelligence

Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM.

llms rag agents reinforcement-learning efficiency multimodal benchmarks safety

arXiv AI

Jul 14

MusicMark: A Robust Generative Watermarking Framework for Music Generation

arXiv:2607. 11117v1 Announce Type: cross Abstract: AI music generation has rapidly advanced alongside commercial platforms, raising the need for reliable watermarking for provenance and attribution.

By Seohwan Yun, Jeeyoung Yun, Yongjin Kim, Juyeon Lee, Sungwoong Kim

rag diffusion

arXiv Machine Learning

1d ago

What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models

arXiv:2608. 14819v1 Announce Type: cross Abstract: Music foundation models are commonly used as frozen audio feature extractors, yet selecting which layer to extract from remains largely heuristic.

By Angelos-Nikolaos Kanatas, Yuexuan Kong, Pablo Alonso-Jim\'enez, Xavier Serra, Dmitry Bogdanov

arXiv AI

Jun 9

BareWave: Waveform-Native Flow-Matching Text-to-Speech

arXiv:2606. 09048v1 Announce Type: cross Abstract: Removing intermediate representations and separately trained decoding stages has become an important direction in generative modeling.

By Wei Fan, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Kejiang Chen, Weiming Zhang, Nenghai Yu

multimodal safety

Related stories