arXiv Machine Learning By Yu Zhou, Changsong Lei, Yixiang Wei, Kairui Wen, Huankun Sheng, Ming Li, Yeying Fan, Yu-Hui Wen, Yong-Jin Liu

Beyond Prediction Gain: LASR as a Metric for Action-Grounded Latents in World Models

Read the original on arXiv Machine Learning →

The paper introduces the Latent Action Support Ratio (LASR), a dimensionless metric designed to evaluate whether latent representations in world models encode action-related information beyond simple predictive shortcuts. LASR offers two summaries—GeoLASR for typical support and WorstLASR for the weakest case—using offline evaluation on held-out data with recorded actions and state changes as targets. Experiments on robot manipulation data and a multi-task panel show that LASR can detect genuine action information, reject non-action shortcuts, and facilitate graded comparisons among imperfect latent representations, also aiding in training and auditing a lightweight residual adapter.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Jun 2

From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data

arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.

By Zhiyuan Feng, Qixiu Li, Huizhi Liang, Rushuai Yang, Yichao Shen, Zhiying Du, Zhaowei Zhang, Yu Deng, Li Zhao, Hao Zhao, Zongqing Lu, Oier Mees, Marc Pollefeys, Jiaolong Yang, Baining Guo
arXiv Computer Vision
Oct 8

Video Prediction Policy 2: Predict Better, Act Better

Video Prediction Policy 2 (VPP2) is a new world action model that improves zero‑shot generalization for both video prediction and action generation. It achieves this by pretraining a large, diverse manipulation video dataset with event‑level supervision, then distilling the model into a single‑step visual planner and adding a mixture‑of‑transformers action module. Experiments show VPP2 outperforms leading baselines on open‑ended video prediction, real‑world zero‑shot manipulation, and several challenging benchmarks.

By Yanjiang Guo, Haodong Yan, Zhide Zhong, Zhongru Zhang, Qingyuan Yang, Qingzhou Lu, Xiaoyu Chen, Yen-Jen Wang, Shuying Deng, Chenghan Yang, Puzhen Yuan, Chenxin Liu, Tun Ban, Xiang Zhu, Yichen Liu, Kun Feng, Haoang Li, Jianyu Chen
arXiv Computer Vision
Oct 15

OneLA: Learning Many Worlds from Latent Actions

arXiv:2610.20160v1 Announce Type: new Abstract: Learning world models across diverse domains calls for a unified control space, yet ac- tion interfaces vary across domains and action annotations are...

By Ziyu Zhu, Zhaoxu Sun, Fan Jiang, Chiyu Wang, Junfeng Ni, Mu Xu
arXiv Computer Vision
6d ago

OnlyAct: Extracting Action Representations from Foundation Models for World Models

OnlyAct proposes extracting action representations from a pretrained foundation model (V-JEPA 2) instead of learning actions from scratch. It uses a label‑free contrastive adaptation framework to isolate motion while discarding scene information, avoiding reconstruction bottlenecks. The method is evaluated on diverse benchmarks, showing more expressive action representations and better transfer of actions to new scenes.

By Runjia Li, Minghao Chen, Junyu Xie, Philip Torr, Andrea Vedaldi, Tomas Jakab
arXiv AI
3d ago

SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models

SALA introduces a semantically aligned latent action framework that bridges human and robot video data without requiring paired examples. By leveraging language descriptions to estimate semantic similarities, SALA guides human latent actions toward related robot latents while grounding the latent space with robot action reconstruction. This shared latent action space enables dual‑axis scaling: larger general human pre‑training reduces robot action prediction error, and task‑specific human post‑training boosts real‑world robot success.

By Jiachun Jin, Zetong Zhou, Haonan Wang, Xiao Yang, Jun Zhu, Zhijie Deng