ViT-H is a video‑to‑humanoid control framework that generates humanoid reference motions directly from monocular human videos using a diffusion model, eliminating the need for separate reconstruction, retargeting, and tracking stages. The system incorporates motion‑tracking feedback to fine‑tune the generator via preference optimization, and an autoregressive variant enables real‑time teleoperation with sub‑400 ms latency. Experiments on the Unitree G1 robot show that ViT‑H surpasses existing reconstruction‑retargeting‑tracking pipelines in motion quality and executability, and its performance scales positively with model size and data volume.
arXiv:2511. 07820v4 Announce Type: replace-cross Abstract: Despite the rise of billion-parameter foundation models trained across thousands of graphical processing units (GPUs), similar scaling gains have not been shown for humanoid control.
BeyondRetarget is an end‑to‑end framework that learns to generate executable humanoid robot motions directly from monocular RGB videos, bypassing the need for an explicit human motion representation. By learning robot‑oriented implicit representations and incorporating a contact‑aware motion optimization mechanism, the method captures cross‑morphology motion structures and improves temporal consistency and physical plausibility. Experiments demonstrate that BeyondRetarget achieves higher execution success rates, lower latency, and greater accuracy and robustness in both simulation and real humanoid robots.
arXiv:2610.34126v1 Announce Type: cross Abstract: Humanoid robots have recently achieved remarkable success in motion tracking. However, during real-world deployment, robots inevitably encounter dyna...
arXiv:2606. 30645v1 Announce Type: cross Abstract: Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion.
arXiv:2603.19305v3 Announce Type: replace-cross Abstract: Humanoid robots are expected to execute agile and expressive whole-body motions in real-world settings. Existing text-to-motion generation mo...
arXiv:2509. 15443v2 Announce Type: replace-cross Abstract: Human-to-humanoid imitation learning presents a promising pathway to address the severe data scarcity bottleneck in robotics by utilizing abundant, large-scale human motion collections.
arXiv:2607. 08741v1 Announce Type: cross Abstract: Generating realistic 3D human motions in real-time within interactive applications is key for animation, simulation, and humanoid robotics.
The paper introduces Movens-5M, a dataset of 15 million human‑motion sequences and 5 million humanoid‑executed motions derived from raw Internet videos, aligned with text, RGB video, speech, and music. Using this data, the authors present Movens‑Zero, a scalable architecture that jointly learns across modalities and supports real‑time closed‑loop execution, enabling motion generation from text, speech, music, and video. They also release Movens‑Intent, a benchmark of 8,192 expert‑verified condition‑motion pairs, and demonstrate that scaling data quality and model capacity improves performance across all modalities.
ULTRA is a unified framework for autonomous humanoid whole-body locomotion and manipulation that overcomes limitations of prior methods by combining a physics-driven neural retargeting algorithm with a multimodal controller. The retargeting algorithm translates large-scale motion capture data into physically plausible humanoid motions, while the controller learns to handle both dense motion references and sparse task specifications using a range of sensory inputs, from accurate motion-capture states to noisy egocentric vision. In simulation and on a real Unitree G1 humanoid, ULTRA demonstrates improved generalization and robustness, enabling coordinated whole-body behavior from sparse intent without relying on test-time reference motions.
Being‑M0.7 is a latent world‑action model that learns from a large mixed‑modality human dataset (over 10,000 hours of video‑only, motion‑only, and paired video‑motion data) to predict future latent visual states and motion. The model is pre‑trained on human data, adapted to robot viewpoints during mid‑training, and then combined with an action expert via gated cross‑attention during post‑training to produce executable whole‑body commands. In experiments, Being‑M0.7 outperforms baselines on the SIMPLE benchmark and matches the best baseline on real‑world Unitree G1 loco‑manipulation tasks.
arXiv:2610.03691v1 Announce Type: new Abstract: We present FlowHMR, a framework for recovering physically plausible global 3D human motion from monocular video. Previous learning-based methods typica...
Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion. Learning this mapping requires synchronized egocentric images, language commands, and robot-compatible kinematic trajectories, yet no existing data source provides this complete tuple at scale.