arXiv AI By Mateusz Modrzejewski, Joanna Kurkowska, Dorien Herremans

A Tutorial on Generative Models for Music Audio

Read the original on arXiv AI →

A tutorial monograph on generative models for music audio covers a wide range of applications—from instrumental clips and full songs with vocals to edits of existing recordings and real‑time accompaniment systems. It explains neural audio codecs, autoencoders, autoregressive and masked token models, diffusion and flow matching, sampling acceleration, and preference optimization, linking training objectives to published systems through worked examples. The book also discusses musical requirements for different tasks, compares representations and methods, and addresses evaluation, data provenance, memorization, attribution, and legal implications.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Oct 8

Tracing Inputs, Verifying Outputs: Validating Attribution in Music Generation

The paper presents a method for attributing AI-generated music to its source audio inputs. By conditioning a music generator, MixAudio, solely on audio and tracing the inputs behind each output, the authors demonstrate that generated stems follow the timbre of the prompt audio and the harmony of the context audio. They also audit memorization with a musical version identification model, musicDNA, finding few reproductions outside the input records and achieving higher precision and recall than other detectors in human-judged cases.

By Taejun Kim, Wonil Kim, Jongmin Jung, Hyeongseok Wi, Sangeun Kum, Keunhyoung Luke Kim, Taehyoung Kim, Dongjoo Moon, Seungsoon Park, Taewan Kim, Virginie Berger, Juhan Nam, Jongpil Lee
arXiv AI
Jul 23

Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering

arXiv:2607. 20253v1 Announce Type: cross Abstract: In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes.

By Junyu Dai, Xinyue Fan, Weiqin Li, Xiangang Li, Yunjia Li, Bin Ma, Yukun Ma, Chongjia Ni, Yufei Shi, Haoxu Wang, Menglin Wu, Jianwei Yu, Huaicheng Zhang, Han Zhao, Shengkui Zhao, Haina Zhu
arXiv AI
Sep 15

DuoTok: Source-Aware Dual-Track Music Tokenization for Vocal-Accompaniment Generation

DuoTok is a source‑aware dual‑track music tokenizer designed for vocal‑accompaniment generation. It first learns a semantic audio representation via self‑supervised pretraining, then refines source‑aware structure with feature‑replacement noise and multi‑task supervision (spectral reconstruction, source separation regularization, and an ASR head for lyric alignment). The encoder is frozen and hard‑routed codebooks for vocals and accompaniment are learned, while a diffusion decoder restores fine acoustic detail from the discrete tokens, achieving a favorable predictability‑fidelity trade‑off at ultra‑low bitrate across public benchmarks.

By Rui Lin, Zhiyue Wu, Jiahe Lei, Kangdi Wang, Weixiong Chen, Junyu Dai, Tao Jiang
Hugging Face Trending Papers
Jul 22

Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering

In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes. The proposed framework supports three tasks: Lyrics-to-Song Generation, which generates complete songs from text descriptions, lyrics, and musical attributes; Instrumental Music Generation, which creates music without vocals; and Cover Song Generation, which reinterprets existing songs with different styles while preserving their melodic content.