arXiv Machine Learning By Jingyan Zhang, Han Liang, Ruichi Zhang, Bin Li, Juze Zhang, Xin Chen, Jingya Wang, Lan Xu, Jingyi Yu

SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Control

Read the original on arXiv Machine Learning →

SCRIPT is a scalable diffusion policy that uses a Joint Action-State-Text Diffusion Transformer (JAST‑DiT) to jointly encode actions, physical states, and natural‑language instructions, enabling direct interaction between language semantics and control dynamics. The method employs a multi‑stage training framework, including supervised imitation pre‑training, a nonlinear history conditioning mechanism for stable autoregressive control, and a post‑training stage with Reinforcement Learning with Hybrid Rewards (RLHR) that injects learnable noise to improve motion quality and instruction following. Experiments on the 1200‑hour MotionMillion dataset show that SCRIPT outperforms prior state‑of‑the‑art methods across text alignment, motion quality, and physical realism, and its performance scales consistently with model size.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computer Vision
Sep 21

ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation

ULTRA is a unified framework for autonomous humanoid whole-body locomotion and manipulation that overcomes limitations of prior methods by combining a physics-driven neural retargeting algorithm with a multimodal controller. The retargeting algorithm translates large-scale motion capture data into physically plausible humanoid motions, while the controller learns to handle both dense motion references and sparse task specifications using a range of sensory inputs, from accurate motion-capture states to noisy egocentric vision. In simulation and on a real Unitree G1 humanoid, ULTRA demonstrates improved generalization and robustness, enabling coordinated whole-body behavior from sparse intent without relying on test-time reference motions.

By Xialin He, Sirui Xu, Xinyao Li, Runpei Dong, Liuyu Bian, Yu-Xiong Wang, Liang-Yan Gui