Music-driven dance video generation aims to synthesize expressive human motion that is temporally aligned with music while maintaining high visual fidelity. Despite recent progress, existing methods still face two key limitations: the lack of large-scale, high-quality dance video datasets, and the absence of principled frameworks for integrating music as a complementary conditioning signal into Video Generation Foundation Models.
SalsaAgent is a multimodal embodied language model that generates expressive, full‑body salsa follower motions in response to a human leader and music. The approach treats partner interaction as nonverbal token passing, extending a large language model’s vocabulary to include discrete motion, pairwise relation, and audio tokens. A two‑stage token‑to‑diffusion pipeline, combined with full‑body and pairwise‑relation tokenizers and alignment with automatically derived text descriptions of skeleton dynamics, yields improved motion quality, spatial coordination, and music‑partner synchrony compared to prior baselines.
By Payam Jome Yazdian, Zoe Stanley, Angelica Lim
arXiv:2607. 13978v1 Announce Type: cross Abstract: Music-driven dance generation aims to produce human motion that is both rhythmically synchronized and semantically consistent with music.
By Xinhao Cai, Yixuan Sun, Minghang Zheng, Qingchao Chen, Xin Jin, Song-chun Zhu, Yang Liu
ChoreoStudio is a music‑driven group choreography system that explicitly plans what formations dancers should adopt, when to transition between them, and which dancers execute each transition. It uses a music disentangler to extract rhythmic and structural features, a frozen omni‑modal model to generate a global semantic script, and a Director module that outputs centroid‑relative formation points, transition gates, and mobility allocations. These high‑level plans guide a diffusion generator that produces beat‑aligned body motion, resulting in state‑of‑the‑art motion quality, group organization, and formation dynamics on the AIOZ‑GDance benchmark.
By Jing Xu, Cunjian Chen, Qiuhong Ke
In recent years, advancements in deep learning and generative models have revolutionized music-driven dance generation. This paper introduces a novel platform, namely DanceDuo, leveraging diffusion models to generate AI-choreographed dance sequences synchronized with a variety of music genres, to encourage dancing practice.
arXiv:2606. 22726v2 Announce Type: replace Abstract: Choreographic motion generation poses unique challenges for AI, demanding precise semantic control over complex, temporally structured, and expressive full-body dynamics.
By Seong Jong Yoo, Siyuan Peng, Felix Gu, Stratis Aloimonos, Cornelia Ferm\"uller
arXiv:2609.37400v1 Announce Type: new
Abstract: Generating realistic 3D dance from music is a challenging task that requires accurate synchronization with musical rhythms while capturing the spatial...
By Xiaojian Shen, Dahu Shi, Jianrong Zhang, Hai Li, Hongwei Zhao, Dawei Zhang, Yunzhi Zhuge, Zhiliang Wu, Guanghui Yue, Wei Zhou
The paper introduces Frequency-Factorized Multi-Rate Tokenization (FFMT) for music-to-dance generation, separating root-relative motion into a low-frequency choreographic scaffold and a high-frequency residual. FFMT assigns a lower token rate and smaller codebook to the low-frequency component, while using a higher token rate and larger codebook for the high-frequency part, enabling a plan-then-articulate autoregressive factorization. Experiments on AIST++ and FineDance demonstrate strong distributional fidelity, realistic diversity, and music–motion synchronization, with ablations confirming the benefits of frequency factorization, asymmetric temporal allocation, and low-to-high conditioning.
By Qirong Guo, Ruijie Li, Zeyan Li, Sicheng Shu, Guanzhi Ding, Weihang Ding
InterSing is a framework that generates realistic 3D head animations for duet singing by modeling the sparse, rhythm‑dependent interactions between performers. It introduces interaction logits—a weakly supervised, interpretable latent representation of cross‑performer engagement—and uses them to condition an interaction‑aware diffusion model driven by audio and interaction dynamics. The approach enables unified multi‑mode generation, producing coordinated behavior, independent motion, and smooth transitions, and it generalizes to multi‑singer performances with intuitive control over engagement.
By Yihan Zhou, Zikai Huang, Yuyang Yu, Xuemiao Xu, Cheng Xu, Shengfeng He
arXiv:2603.08590v4 Announce Type: replace
Abstract: Text-to-motion generation has advanced with larger corpora and stronger generators, yet many models still rely on holistic frame- or clip-level lat...
By Zeyu Ling, Qing Shuai, Teng Zhang, Shiyang Li, Bo Han, Changqing Zou
UniMate is a unified foundation model that generates articulated motion for any skeleton from a rigged 3D asset and a text prompt, eliminating the need for test‑time optimization or per‑skeleton retraining. It uses a topology‑aware diffusion transformer that incorporates skeletal topology through graph‑aware attention bias, spectral rotary position embedding, and a global topological conditioner. Trained on the newly curated UniML3D dataset of 13,006 diverse motion sequences, UniMate outperforms existing baselines in quality, generalization, and efficiency, and supports zero‑shot cross‑topology transfer, in‑betweening, expansion, and text‑guided editing.
By Linzhan Mou, Jiahui Lei, Zhiyang Dou, Chenyue Cai, Chaoyue Song, Adam Finkelstein, Szymon Rusinkiewicz
arXiv:2606. 30266v1 Announce Type: cross Abstract: Motion-language agents must possess the bidirectional capability to both understand human movement (motion-to-text, M2T) and generate it from natural language (text-to-motion, T2M).
By Bertram Taetz, Hugo Albuquerque Cosme da Silva, Gabriele Bleser-Taetz