DuET is a Dual-Encoder Transformer that jointly learns note-level piano performance-to-score transcription and frame-level beat/downbeat tracking. It uses bidirectional cross‑attention between note and frame encoders and an autoregressive decoder that alternates between their representations to predict structured score attributes. The model introduces TimeRoPE and Time‑Pitch Rotary Positional Encoding (TPRoPE) to share a performance‑time reference and encode relative temporal and pitch relationships, achieving significant error reductions on ASAP and Gangqinpu datasets.
By Wei Zeng, Ye Wang
Piano2Pop is a music‑expansion task that converts piano MIDI into full instrumental pop productions. The method uses a pseudo‑supervised approach that conditions a pretrained flow‑matching model on piano rolls, trained on synthetic piano‑instrumental pairs derived from pop recordings. Experiments on 476 real‑world piano‑cover/pop pairs show that a pair‑supervised piano‑cover generator outperforms zero‑shot baselines, while function‑selective transcription (FST) reduces piano‑heavy outputs in pair‑free training and remains a promising alternative.
By Yun-Chen Cheng, Chih-Pin Tan, Tzu-Hung Huang, Yi-Hsuan Yang
arXiv:2606. 10627v1 Announce Type: cross Abstract: The quality of piano performance depends on nuanced timing, articulation, and dynamic control, but practice feedback is often summary-based and hard to act on.
By Kazuki Kawamura, Fujiki Nakamura, Hayato Nishioka, Momoko Shioki, Shinichi Furuya, Jun Rekimoto
arXiv:2606. 12282v1 Announce Type: cross Abstract: Expressive performance rendering (EPR) aims to generate realistic performances constrained on sequences of notes.
By Dmitrii Gavrilev
arXiv:2604.15086v3 Announce Type: replace-cross
Abstract: Recent advances in video-to-audio (V2A) generation enable high-quality audio synthesis from visual content, yet achieving robust and fine-gra...
By Jianxuan Yang, Xinyue Guo, Zhi Cheng, Kai Wang, Lipan Zhang, Jinjie Hu, Qiang Ji, Yihua Cao, Yihao Meng, Zhaoyue Cui, Mengmei Liu, Meng Meng, Jian Luan
PVSync is a unified model that estimates audio‑visual offset and scores phoneme‑level articulation for lip‑sync tasks. It uses window‑level contrastive learning for synchronisation and a phoneme‑level articulation objective that aligns audio and video embeddings of the same viseme class across clips. The model achieves higher correlation with human rankings and better performance on offset recovery and viseme matching benchmarks compared to existing methods.
By Kevin Stephen, Varun Menon, Timo Mertens, Nikita Drobyshev