Hugging Face Trending Papers

SMEBENCH: Benchmarking Neural Music Editing with Synthetized MIDI Edits

arXiv AI
Aug 20

Evaluating Music Context Preservation: A Multi-facet Framework for Music Editing Systems

The paper introduces MuseCPEval, the first comprehensive framework for assessing Music Context Preservation (MuseCP) in music editing systems. It defines four categories of music facets and provides fine‑grained metrics to detect subtle changes during editing tasks such as timbre transfer, instrument substitution, and genre transformation. The authors validate the metrics objectively and through a human study, and demonstrate their practical use in evaluating diverse editing systems, offering insights into each system’s strengths and limitations.

By Yash Vishe, Eric Xue, Xunyi Jiang, Zachary Novack, Junda Wu, Julian McAuley, Xin Xu
arXiv AI
2d ago

Gangqinpu: A Multi-Genre, Multi-Task Piano Dataset with Aligned Audio, MIDI, and Musical Scores

Gangqinpu is a large, multi‑genre piano dataset that provides aligned audio, MIDI, and musical scores for 18,786 pieces, covering pop, classical, jazz, folk/traditional, rock, and blues. The dataset includes automatic note‑level alignments, beat/downbeat annotations, genre labels, and a binary classifier to filter non‑expressive performances. It is designed to reduce data leakage with ASAP splits and has shown cross‑dataset improvements in beat tracking and in‑domain gains for transcription and expressive performance rendering.

By Wei Zeng, Anvay Mathur, Ye Wang
arXiv Machine Learning
Sep 23

Synthesis and editing of multi-instrument audio mixtures using scalar-quantised latents with MIDI Span conditioning

SpanSynth-Edit is a flow‑matching model that enables MIDI‑guided synthesis and editing of multi‑instrument audio mixtures using low‑frame‑rate scalar‑quantised latents. It encodes instrument‑labelled note lifecycles as unordered event sets, pools them into a conditioning vector per audio‑latent frame, and uses contextual audio for instrument‑specific timbre guidance. The model supports editing by resynthesising target regions from revised MIDI and demonstrates competitive performance on single‑ and multi‑instrument benchmarks, including within‑frame onset control.

By Sungkyun Chang, Keshav Bhandari, Simon Dixon, Emmanouil Benetos
arXiv Computer Vision
Sep 7

AVENUE: Audio-Video EditiNg Understanding and Evaluation

AVENUE is a new benchmark and evaluation framework for audio‑video editing that includes 1,291 source clips and 7,957 editing instructions covering audio‑targeted, video‑targeted, and coupled edits. It introduces a sample‑specific, modality‑aware evaluation that specifies the intended change and the content that must remain unchanged. The study applies this framework to joint, sequential, and separate editing models, revealing that existing models often alter unintended modalities, highlighting a key challenge in controllable AV editing.

By Hayeon Kim, Yoojin Jang, Jaejun Yoo
arXiv AI
Jul 14

A Production-Oriented Framework for Evaluation of SFX Generation

arXiv:2607. 09973v1 Announce Type: cross Abstract: Industrial sound design requires audio generation systems that not only produce realistic audio, but also preserve the perceptual identity of a reference, support controllable variation, and remain efficient for practical workflows.

By M\'elodie Desbos, Yara Bahram, Eric Granger, Mohammadhadi Shateri
arXiv AI
Jun 26

Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training

arXiv:2512. 02652v2 Announce Type: replace-cross Abstract: Existing methods for expressive music performance rendering, a conditional generation task that aims to generate a human-like performance from a symbolic score, rely on supervised learning over small labeled datasets, which limits scaling of both data volume and model size, despite the availability of vast unlabeled music, as in vision and language.

By Hong-Jie You, Jie-Jing Shao, Xiao-Wen Yang, Lin-Han Jia, Lan-Zhe Guo, Yu-Feng Li
arXiv AI
6d ago

Agentic Music Post-Production

The paper introduces MixAgent, an agentic pipeline for music post‑production that emulates audio engineers’ workflow to create interpretable and editable mixing plans guided by a reference song. It employs a grouping agent to cluster related tracks, a planning agent to select and order audio effects, and a detector‑critic pair to refine the mix by identifying needed corrections and proposing parameter adjustments. The authors also present MixCritic, a dataset of 45,657 repair critiques, and demonstrate that supervised fine‑tuning followed by preference optimization improves mix quality, achieving a 19% error reduction on MedleyDB and outperforming several large language models in human listening tests.

By Jaywon Koo, Junghyun Koo, Marco A. Martinez-Ramirez, Wei-Hsiang Liao, Seungheon Doh, Qiyu Wu, Vicente Ordonez, Yuki Mitsufuji
arXiv Machine Learning
Oct 16

MIDI-REPA: Improving Text-to-Music Generation with Symbolic Representation Alignment

MIDI-REPA is a framework that aligns symbolic music representations with intermediate layers of a flow-matching DiT model to improve text-to-music generation. By deriving MIDI supervision from automatic music transcription and aligning a MuScriptor-derived piano roll with an early DiT layer throughout the flow trajectory, the method achieves competitive performance on the ATTM objective metrics without extra training data or techniques. The approach is trainable on a single RTX 3090 in about 31 hours and is supported by released transcription pipeline, training code, and audio samples.

By Song-Ze Yu, Fang-Duo Tsai, Yi-Hsuan Yang
arXiv AI
Jul 16

From Prediction to Collaboration: Interactive Symbolic Music Analysis

arXiv:2607. 13587v1 Announce Type: cross Abstract: Automatic symbolic music analysis has made substantial progress, yet existing systems are typically designed for a single mode of use, such as full-score prediction, and therefore do not match the broader range of operations that arise in analysis workflows, including partial completion, local correction, and iterative refinement.

By Emmanouil Karystinaios, Johannes Hentschel, Markus Neuwirth, Gerhard Widmer
arXiv AI
Sep 2

TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data

TUTTI is a new pre‑training framework for audio‑to‑score transcription that uses a large, fully synthetic multi‑instrument dataset generated by a symbolic music model. The approach trains a standard Transformer encoder‑decoder on these synthetic audio‑score pairs, producing a stronger foundational representation than single‑instrument training. When fine‑tuned on real datasets, TUTTI surpasses prior methods, achieving state‑of‑the‑art results and demonstrating strong cross‑instrument transferability.

By Jianhuai Hu, Yashan Wang, Shangda Wu, Zhancheng Guo, Shijie Liang, Wuna Meng, Chuanqi Yang, Xiaobing Li, Feng Yu, Maosong Sun