← Back to all news
arXiv AI By Jakub Po\'cwiardowski, Mateusz Modrzejewski

MI-MIDI 2: From Observation to Control with Sparse Autoencoders in Text-to-MIDI Generation

Read the original on arXiv AI →

The paper introduces MI-MIDI 2, an end‑to‑end protocol for discovering and evaluating sparse autoencoder (SAE) features in text‑to‑MIDI generation. It applies the protocol to the text2midi and MIDI‑LLM models, characterizing SAE features through correlations with musical metrics, instrument programs, and automated descriptions, and then testing them via direct intervention in generated MIDI. The study finds that certain SAE features—particularly those linked to instrument programs and note‑density adjustments—effectively steer the output, with evidence validated across multiple controls and independent prompts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 23

Synthesis and editing of multi-instrument audio mixtures using scalar-quantised latents with MIDI Span conditioning

SpanSynth-Edit is a flow‑matching model that enables MIDI‑guided synthesis and editing of multi‑instrument audio mixtures using low‑frame‑rate scalar‑quantised latents. It encodes instrument‑labelled note lifecycles as unordered event sets, pools them into a conditioning vector per audio‑latent frame, and uses contextual audio for instrument‑specific timbre guidance. The model supports editing by resynthesising target regions from revised MIDI and demonstrates competitive performance on single‑ and multi‑instrument benchmarks, including within‑frame onset control.

By Sungkyun Chang, Keshav Bhandari, Simon Dixon, Emmanouil Benetos
benchmarks
More like this →
arXiv Machine Learning
Oct 16

MIDI-REPA: Improving Text-to-Music Generation with Symbolic Representation Alignment

MIDI-REPA is a framework that aligns symbolic music representations with intermediate layers of a flow-matching DiT model to improve text-to-music generation. By deriving MIDI supervision from automatic music transcription and aligning a MuScriptor-derived piano roll with an early DiT layer throughout the flow trajectory, the method achieves competitive performance on the ATTM objective metrics without extra training data or techniques. The approach is trainable on a single RTX 3090 in about 31 hours and is supported by released transcription pipeline, training code, and audio samples.

By Song-Ze Yu, Fang-Duo Tsai, Yi-Hsuan Yang
benchmarkssafety
More like this →
arXiv AI
Sep 17

CPR: Combining global composing, local performing and full-sequence refining in piano rendering with continuous autoregressive modelling

The paper introduces CPR, a piano rendering framework that combines continuous autoregressive modeling with local flow matching and full‑sequence refinement. It predicts continuous hidden states, generates 24 kHz acoustic latents, and upsamples to 48 kHz, while new techniques BREPA and MT‑RoPE enhance musical semantics and cross‑modal alignment.

By Chong Jing, Junan Zhang, Zhizheng Wu
diffusionefficiencysafety
More like this →