← Back to all news
arXiv Computer Vision

From Digital Human Interactions to Physics-Based Humanoid Skills: Physics-Grounded Post-Training of Interaction Generators

The paper introduces DIGHT, a co‑adaptive framework that couples a Digital Human Interaction Generator with a Humanoid Tracking policy. DIGHT first runs multiple text‑conditioned interaction candidates in simulation, then builds physics‑grounded preferences from the rollouts—covering executability and interaction fidelity—and uses diffusion‑based direct preference optimization to align the generator without back‑propagating through the simulator. The aligned generator supplies reference motions for fine‑tuning the tracker, leading to more physically plausible and faithful humanoid interactions in simulation.

Related stories

arXiv AI
Oct 9

PhysMoDPO: Physically-Plausible Humanoid Motion with Preference Optimization

PhysMoDPO is a Direct Preference Optimization framework that integrates a Whole-Body Controller into the training of diffusion models for text-conditioned human motion generation. By using physics-based and task-specific rewards to assign preferences to synthesized trajectories, it ensures that the resulting motions are both physically realistic and faithful to the original text instructions. Experiments on simulated robots and a real G1 humanoid robot show consistent improvements in physical realism and task performance, including zero‑shot motion transfer.

By Yangsong Zhang, Anujith Muraleedharan, Rikhat Akizhanov, Abdul Ahad Butt, G\"ul Varol, Pascal Fua, Fabio Pizzati, Ivan Laptev
diffusionrobotics
More like this →
arXiv Machine Learning
Jul 31

REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning

arXiv:2603. 13707v3 Announce Type: replace-cross Abstract: Humanoid loco-manipulation requires coordinated task-space motion planning with stable loco-manipulation command tracking under complex robot-environment dynamics and long-horizon tasks.

By Zhaoyuan Gu, Yipu Chen, Zimeng Chai, Alfred Cueva, Thong Nguyen, Yifan Wu, Huishu Xue, Minji Kim, Isaac Legene, Fukang Liu, KyoungMok Kim, Ayan Barula, Yongxin Chen, Ye Zhao
diffusionreinforcement-learningroboticsfine-tuning
More like this →
arXiv Machine Learning
Sep 7

SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Control

SCRIPT is a scalable diffusion policy that uses a Joint Action-State-Text Diffusion Transformer (JAST‑DiT) to jointly encode actions, physical states, and natural‑language instructions, enabling direct interaction between language semantics and control dynamics. The method employs a multi‑stage training framework, including supervised imitation pre‑training, a nonlinear history conditioning mechanism for stable autoregressive control, and a post‑training stage with Reinforcement Learning with Hybrid Rewards (RLHR) that injects learnable noise to improve motion quality and instruction following. Experiments on the 1200‑hour MotionMillion dataset show that SCRIPT outperforms prior state‑of‑the‑art methods across text alignment, motion quality, and physical realism, and its performance scales consistently with model size.

By Jingyan Zhang, Han Liang, Ruichi Zhang, Bin Li, Juze Zhang, Xin Chen, Jingya Wang, Lan Xu, Jingyi Yu
llmsagentsdiffusionreinforcement-learningroboticsbenchmarkssafety
More like this →
arXiv Computer Vision
Sep 21

ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation

ULTRA is a unified framework for autonomous humanoid whole-body locomotion and manipulation that overcomes limitations of prior methods by combining a physics-driven neural retargeting algorithm with a multimodal controller. The retargeting algorithm translates large-scale motion capture data into physically plausible humanoid motions, while the controller learns to handle both dense motion references and sparse task specifications using a range of sensory inputs, from accurate motion-capture states to noisy egocentric vision. In simulation and on a real Unitree G1 humanoid, ULTRA demonstrates improved generalization and robustness, enabling coordinated whole-body behavior from sparse intent without relying on test-time reference motions.

By Xialin He, Sirui Xu, Xinyao Li, Runpei Dong, Liuyu Bian, Yu-Xiong Wang, Liang-Yan Gui
agentsdiffusionreinforcement-learningroboticsfine-tuningmultimodal
More like this →
arXiv AI
Sep 16

SafeFlow: Real-Time Text-Driven Humanoid Whole-Body Control via Physics-Guided Rectified Flow and Selective Safety Gating

SafeFlow is a real‑time, text‑driven humanoid control framework that blends physics‑guided motion generation with a three‑stage safety gate. It uses Physics‑Guided Rectified Flow Matching in a VAE latent space to produce physically executable trajectories, accelerates sampling with Reflow, and filters unsafe outputs via semantic OOD detection, directional sensitivity checks, and hard kinematic constraints before handing them to a motion‑tracking controller. Experiments on the Unitree G1 show that SafeFlow achieves higher success rates, better physical compliance, and faster inference than diffusion‑ and retargeting‑based baselines while maintaining motion diversity.

By Hanbyel Cho, Sang-Hun Kim, Jeonguk Kang, Donghan Koo
ragdiffusionroboticsefficiencysafety
More like this →
arXiv Machine Learning
Oct 12

HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videos

HumanX is a unified framework that transforms human videos into generalizable, real‑world interaction skills for humanoid robots without task‑specific design. It combines XGen, a data‑generation pipeline that creates physically plausible robot interaction data from monocular video, with XMimic, a learning framework that extracts interaction priors from only seconds of demonstration. Across five domains—basketball, football, badminton, cargo pickup, and reactive fighting—HumanX learns nine distinct skills and transfers them zero‑shot to a physical Unitree G1 humanoid, achieving complex maneuvers and sustained passing sequences from a single video.

By Yinhuai Wang, Qihan Zhao, Yuen Fui Lau, Runyi Yu, Hok Wai Tsui, Qifeng Chen, Jingbo Wang, Jiangmiao Pang, Ping Tan
robotics
More like this →