LAPT-Scale introduces a latent action pre‑training framework that leverages large‑scale egocentric human videos to train embodied robotic policies. The method comprises LAPT‑Scale‑LAM, which learns continuous latent actions by modeling visual transitions in human videos, and LAPT‑Scale‑Policy, which aligns these latent actions with robot embodiment and fine‑tunes for specific tasks. Using 23.3K hours of video, the approach improves real‑robot success rates from 63.3% to 85.0% and generalizes to unseen visual conditions.
By Zhiyuan Xiang, Peidong Liu, Mingyang Li, Jiawei Li, Wenhow Li, Wenbo Ding, Xiang Deng
The paper introduces NAVA-WAM, a method that learns action priors directly from observation-only videos to pretrain action policies for robots. It uses a two-stage training process: first, it pretrains on videos with future‑video flow‑matching supervision to learn action‑relevant priors, and second, it fine‑tunes with action‑labeled demonstrations for robot control. Experiments show that NAVA‑WAM outperforms previous methods in both in‑distribution and out‑of‑distribution scenarios, achieving strong action‑label efficiency and real‑robot generalization.
By Zhaochong An, Fei Zhang, Menglin Jia, Duncan Frost, Zijian Zhou, Yikai Wang, Xudong Wang, Aditya Patel, Belinda Zeng, Tao Xiang, Serge Belongie, Amir Bar, Sen He
arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.
By Zhiyuan Feng, Qixiu Li, Huizhi Liang, Rushuai Yang, Yichao Shen, Zhiying Du, Zhaowei Zhang, Yu Deng, Li Zhao, Hao Zhao, Zongqing Lu, Oier Mees, Marc Pollefeys, Jiaolong Yang, Baining Guo
The paper introduces P³-WAM, a unified world action model for egocentric dexterous manipulation that integrates 3D perception, action-conditioned prediction, and goal-directed planning. It operates in three modes—Perceive, Predict, and Plan—using a mixture-of-transformers architecture that shares base weights and adapts to modality-specific inputs. Experiments demonstrate that unified training improves performance across all three capabilities, outperforming specialized baselines on 3D hand reconstruction and action-conditioned video prediction while producing more instruction‑adherent manipulation videos.
By Zirui Wu, Chen Guo, Tianjian Jiang, Zeren Jiang, Zeming Li, Lingchen Yang, Longteng Duan, Ying Xue, Handi Yin, Shenyuan Gao, Martin R. Oswald, Jun Zhang, Ping Tan, Jie Song
arXiv:2610.16443v1 Announce Type: new
Abstract: Recent world action models draw on video generation priors to jointly predict future observations and robot actions. Adapting these priors to control r...
By Yuanpeng He, Ceyao Zhang, Fangjing Li, Kefei Zhu, Lijian Li, Yueheng Li, Tianjia He, Wenpin Jiao, Zhi Jin, Yaodong Yang, Yuanpei Chen
The paper introduces HF‑WM, an action‑conditioned world model that learns from large‑scale egocentric human videos and incorporates failure dynamics through post‑training. It bridges the embodiment gap with a dual‑branch framework that couples video generation to stabilized pointflow prediction, unifying motion representation across humans and robots while reducing egocentric camera motion. Failure modeling is enhanced by Group Relative Policy Optimization on perturbed actions, guided by action alignment and physical plausibility rewards, enabling the model to anticipate unsuccessful executions. Experiments show HF‑WM outperforms existing methods in action‑conditioned video generation and is useful for evaluating policies in downstream robotic tasks.
By Peiyan Tu, Cong Wang, Hanxin Zhu, Xin Jin, Zhibo Chen
Video Prediction Policy 2 (VPP2) is a new world action model that improves zero‑shot generalization for both video prediction and action generation. It achieves this by pretraining a large, diverse manipulation video dataset with event‑level supervision, then distilling the model into a single‑step visual planner and adding a mixture‑of‑transformers action module. Experiments show VPP2 outperforms leading baselines on open‑ended video prediction, real‑world zero‑shot manipulation, and several challenging benchmarks.
By Yanjiang Guo, Haodong Yan, Zhide Zhong, Zhongru Zhang, Qingyuan Yang, Qingzhou Lu, Xiaoyu Chen, Yen-Jen Wang, Shuying Deng, Chenghan Yang, Puzhen Yuan, Chenxin Liu, Tun Ban, Xiang Zhu, Yichen Liu, Kun Feng, Haoang Li, Jianyu Chen
arXiv:2609.22332v1 Announce Type: cross
Abstract: Generalizable robot manipulation requires predicting how a scene will evolve, identifying where interactions are feasible, and determining how to act...
By Jiadi You, Qize Yu, Yue Chen, Minghong Cai, Zhide Zhong, Yuran Wang, Bowen Ping, Jiaqi Liang, Zhenhao Shen, Haodong Yan, Yinchuan Li, Ruihai Wu, Xiaojuan Qi, Yingcong Chen
arXiv:2610.28122v1 Announce Type: cross
Abstract: Cascade world-action models (WAMs) use predicted visual futures to condition robot actions, but inherit costly iterative inference from diffusion-pre...
By Shaoyi Zheng, Zhongtao Jiang, Yue Peng, Wenbo Lu, Shunchi Zhang, Xi Wang, Shengji Wan
Generalizable robot manipulation requires policies that can anticipate how visual scenes evolve while executing language instructions. While recent Vision-Language-Action models benefit from large-scale pretraining, their predominantly static pretraining objectives provide limited supervision for physical dynamics and temporal causality, leaving control-relevant knowledge to be learned from downstream robot demonstrations.
arXiv:2610.33685v1 Announce Type: cross
Abstract: Vision-Language-Action (VLA) models are bottlenecked by the scarcity of robot data. Egocentric human manipulation videos offer an abundant, diverse,...
By Qingyuan Cai, Bo Zhang, Yongzhen Huang, Jianyu Chen
arXiv:2610.20604v1 Announce Type: new
Abstract: World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined in...
By Ao Liang, Youquan Liu, Mingxiao Li, Wei Chow, Lingdong Kong, Yue Liao, Dongyue Lu, Yu Yang, Wei Yin, Runnan Chen