arXiv Machine Learning

Project Qualia: Recovering Experiential Music Structure from Session Co-occurrence Data

Project Qualia investigates whether experiential similarity between songs can be extracted from listening behavior. Using 1.29 billion scrobbles from 9,396 users, the authors trained a Word2Vec model (Song2Vec) on session data, then applied an artist‑residual procedure to isolate artist‑independent signals. The residual embeddings still contained strong cross‑artist similarity, forming coherent genre and era clusters, demonstrating that experiential structure exists beyond artist identity.

arXiv AI
Sep 16

MUUNRiver-Bench: Diagnosing Relation-Dependent Music Retrieval with Multimodal Instructions

MUUNRiver-Bench is a diagnostic benchmark for music retrieval that uses natural‑language instructions to define relevance for reference‑audio queries. It contains 3,440 tracks across 13 genres and 116 sub‑genres and covers seven tasks such as similar‑music, style‑preserving lyric‑rewriting, cover, and segment retrieval. Experiments with six models in eight configurations show that acoustic encoders favor local identity while text‑aligned encoders favor semantic relations, and that instruction‑aware and audio‑text fusion systems do not consistently outperform their backbones.

By Zhancheng Guo, Congren Dai, Shangda Wu, Jianhuai Hu, Danni Zhao, Xiaobing Li, Maosong Sun
arXiv Machine Learning
Aug 28

Recovering Expert Critic-Sourced Network Adjacency between Musical Artists from Acoustic Distributions: A Construct-Validity Approach

The paper investigates whether expert critic‑sourced relationships between musical artists—called critical adjacency—can be recovered from the artists’ acoustic content. By modeling artists as distributions over 80 low‑level acoustic descriptors and measuring pairwise proximity with Wasserstein distances, the authors achieve an out‑of‑sample AUC of 0.767 for predicting critic‑linked edges, with higher recoverability for edges supported by multiple critics. The results suggest that critical discourse contains a reproducible sonic core, while sociological factors also influence the perceived adjacency between artists.

By Elena Badillo-Goicoechea, Fengfeng He
arXiv AI
Sep 2

On the Human and Computer Alignment of Attribute-Based Music Matches

The paper introduces the MATCHA dataset, comprising 1,105 perceptual assessments from 83 experts on attribute-based music matches across five musical attributes—melody, harmony, rhythm, voice, and timbre. A triplet-based forced-choice experiment with 300 cases, including plagiarism, cover songs, and AI-generated music, was used to gather these judgments. Results show measurable agreement among participants and partial alignment with computational similarity measures, highlighting the need for perceptually grounded evaluation in generative AI for music.

By Roser Batlle-Roca, Woosung Choi, Joan Serr\`a, Fabio Morreale, Wei-Hsiang Liao, Xavier Serra, Emilia G\'omez, Yuki Mitsufuji
arXiv AI
4d ago

A Tutorial on Generative Models for Music Audio

A tutorial monograph on generative models for music audio covers a wide range of applications—from instrumental clips and full songs with vocals to edits of existing recordings and real‑time accompaniment systems. It explains neural audio codecs, autoencoders, autoregressive and masked token models, diffusion and flow matching, sampling acceleration, and preference optimization, linking training objectives to published systems through worked examples. The book also discusses musical requirements for different tasks, compares representations and methods, and addresses evaluation, data provenance, memorization, attribution, and legal implications.

By Mateusz Modrzejewski, Joanna Kurkowska, Dorien Herremans
arXiv AI
Oct 27

Gangqinpu: A Multi-Genre, Multi-Task Piano Dataset with Aligned Audio, MIDI, and Musical Scores

Gangqinpu is a large, multi‑genre piano dataset that provides aligned audio, MIDI, and musical scores for 18,786 pieces, covering pop, classical, jazz, folk/traditional, rock, and blues. The dataset includes automatic note‑level alignments, beat/downbeat annotations, genre labels, and a binary classifier to filter non‑expressive performances. It is designed to reduce data leakage with ASAP splits and has shown cross‑dataset improvements in beat tracking and in‑domain gains for transcription and expressive performance rendering.

By Wei Zeng, Anvay Mathur, Ye Wang