← Back to all news
arXiv Computer Vision By Zimo He, Tianyi Bian, Zizhong Yang, Hongjie Li, Nan Jiang, Jieming Cui, Baoxiong Jia, Yizhou Wang, Siyuan Huang, Yixin Zhu

ViT-H: Generative Video-To-Humanoid Control with Motion Tracking Feedback

Read the original on arXiv Computer Vision →

ViT-H is a video‑to‑humanoid control framework that generates humanoid reference motions directly from monocular human videos using a diffusion model, eliminating the need for separate reconstruction, retargeting, and tracking stages. The system incorporates motion‑tracking feedback to fine‑tune the generator via preference optimization, and an autoregressive variant enables real‑time teleoperation with sub‑400 ms latency. Experiments on the Unitree G1 robot show that ViT‑H surpasses existing reconstruction‑retargeting‑tracking pipelines in motion quality and executability, and its performance scales positively with model size and data volume.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Aug 14

SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control

arXiv:2511. 07820v4 Announce Type: replace-cross Abstract: Despite the rise of billion-parameter foundation models trained across thousands of graphical processing units (GPUs), similar scaling gains have not been shown for humanoid control.

By Zhengyi Luo, Ye Yuan, Tingwu Wang, Chenran Li, Fernando Casta\~neda, Sirui Chen, Zi-Ang Cao, Jiefeng Li, David Minor, Qingwei Ben, Jinhyung Park, David Sami, Zi Wang, Xingye Da, Runyu Ding, Cyrus Hogg, Lina Song, Edy Lim, Eugene Jeong, Tairan He, Haoru Xue, Wenli Xiao, Simon Yuen, Jan Kautz, Yan Chang, Umar Iqbal, Linxi "Jim" Fan, Yuke Zhu
agentsroboticsmultimodal
More like this →
arXiv Computer Vision
Sep 25

BeyondRetarget: Learning Executable Humanoid Motions Directly from Monocular Video

BeyondRetarget is an end‑to‑end framework that learns to generate executable humanoid robot motions directly from monocular RGB videos, bypassing the need for an explicit human motion representation. By learning robot‑oriented implicit representations and incorporating a contact‑aware motion optimization mechanism, the method captures cross‑morphology motion structures and improves temporal consistency and physical plausibility. Experiments demonstrate that BeyondRetarget achieves higher execution success rates, lower latency, and greater accuracy and robustness in both simulation and real humanoid robots.

By Tianyu Xiong, Yi Lu, Jinrui Wang, Ziqi Liang, Dandan Lei, Xiaoyang Zhou, Xiao-xiao Long, Qiu Shen, Xun Cao
robotics
More like this →
arXiv AI
Jul 1

A Scalable Whole-body Motion Transfer via Implicit Kinodynamic Motion Retargeting

arXiv:2509. 15443v2 Announce Type: replace-cross Abstract: Human-to-humanoid imitation learning presents a promising pathway to address the severe data scarcity bottleneck in robotics by utilizing abundant, large-scale human motion collections.

By Xingyu Chen, Hanyu Wu, Sikai Wu, Mingliang Zhou, Diyun Xiang, Haodong Zhang, Yangchen Zhou, Yukang Gao, Yi Gu, Renjing Xu
diffusionroboticsefficiencysafety
More like this →