The paper explains why GPT‑style language models fail to transfer directly to symbolic music. It argues that success in language comes from tokenization that compresses data by creating a coordinate system where recurring patterns become predictable. For music, the authors propose that tokenization must build a predictively effective, relationally lossless coordinate system—defining Fact–Token and Token–State boundaries—to enable compression without sacrificing contextual freedom. Controlled experiments confirm that proper coordinate construction improves predictive compressibility, whereas mere sequence compaction does not.
By Yi Wang
arXiv:2607. 01849v1 Announce Type: cross Abstract: Musical performance involves executing a set of high-level musical instructions, yet recovering those instructions from the performance is a challenging inverse problem.
By Yewon Kim, Apurva Gandhi, David Chung, Graham Neubig, Chris Donahue
The paper introduces Whole-Piece Training for Symbolic Music Language Models using Full-Horizon Compressed Recurrence (FHCR), which maintains the full temporal horizon of recurrent memory while compressing its key-value representation to fit GPU limits. An evaluation diagnostic, KV-Reset Context Utilization (KRCU), demonstrates that full-horizon models retain long-range context beyond local windows, whereas limiting recurrent memory weakens this dependence. FHCR thus preserves long-range context utilization while significantly reducing recurrent memory cost, enabling efficient whole-piece modeling.
By Yungang Yi, Weihua Li, Matthew Kuo, Catherine Shi, Quan Bai
The paper proposes the Effectiveness–Losslessness Framework to guide tokenization in domains beyond language, using predictive codelength as a criterion. It introduces two boundaries: the Fact–Token Boundary, where observable structure should be encoded into tokens, and the Token–State Boundary, where context‑dependent relations should remain for model state rather than being pre‑tokenized. Experiments on symbolic music show that making musical time explicit and applying tonal‑frame canonicalization improve predictive performance, while fixed pitch coordinates and reversible BPE can increase predictive code length, indicating that carrier compaction alone does not guarantee better predictions.
By Yi Wang
arXiv:2606. 30642v1 Announce Type: cross Abstract: Full-length song generation must preserve coherence and musicality, render detailed vocal and accompaniment acoustics, and follow lyrics and prompts.
By Shun Lei, Huaicheng Zhang, Dapeng Wu, Yaoxun Xu, Lishi Zuo, Wei Tan, Hangting Chen, Guangzheng Li, Jianwei Yu, Zhiyong Wu, Dong Yu
arXiv:2607. 14537v1 Announce Type: cross Abstract: Rich internal representations of musical structure are essential for music understanding tasks such as machine-assisted music co-writing, yet self-supervised approaches for symbolic music representation remain underexplored, particularly those that encode the hierarchical multiscale nature of musical structures.
By Scott H. Hawley