arXiv AI By Jiachun Jin, Zetong Zhou, Haonan Wang, Xiao Yang, Jun Zhu, Zhijie Deng

SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models

Read the original on arXiv AI →

SALA introduces a semantically aligned latent action framework that bridges human and robot video data without requiring paired examples. By leveraging language descriptions to estimate semantic similarities, SALA guides human latent actions toward related robot latents while grounding the latent space with robot action reconstruction. This shared latent action space enables dual‑axis scaling: larger general human pre‑training reduces robot action prediction error, and task‑specific human post‑training boosts real‑world robot success.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
4d ago

SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models

SALA introduces a method for aligning latent actions across human and robot videos by leveraging language descriptions to create semantic similarities, guiding human latent actions toward related robot latents while grounding the space with robot action reconstruction. This approach avoids the need for paired data and improves training efficiency by preventing visual clustering of latent actions. The shared latent action space enables dual‑axis scaling: larger general human video pre‑training reduces robot action prediction error, and task‑specific human video post‑training boosts real‑world robot task success.

arXiv AI
Oct 12

LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos

LAPT-Scale introduces a latent action pre‑training framework that leverages large‑scale egocentric human videos to train embodied robotic policies. The method comprises LAPT‑Scale‑LAM, which learns continuous latent actions by modeling visual transitions in human videos, and LAPT‑Scale‑Policy, which aligns these latent actions with robot embodiment and fine‑tunes for specific tasks. Using 23.3K hours of video, the approach improves real‑robot success rates from 63.3% to 85.0% and generalizes to unseen visual conditions.

By Zhiyuan Xiang, Peidong Liu, Mingyang Li, Jiawei Li, Wenhow Li, Wenbo Ding, Xiang Deng
arXiv AI
Jun 2

From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data

arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.

By Zhiyuan Feng, Qixiu Li, Huizhi Liang, Rushuai Yang, Yichao Shen, Zhiying Du, Zhaowei Zhang, Yu Deng, Li Zhao, Hao Zhao, Zongqing Lu, Oier Mees, Marc Pollefeys, Jiaolong Yang, Baining Guo
arXiv Computer Vision
Oct 5

Native Action-Prior Learning from Videos for World Action Models

The paper introduces NAVA-WAM, a method that learns action priors directly from observation-only videos to pretrain action policies for robots. It uses a two-stage training process: first, it pretrains on videos with future‑video flow‑matching supervision to learn action‑relevant priors, and second, it fine‑tunes with action‑labeled demonstrations for robot control. Experiments show that NAVA‑WAM outperforms previous methods in both in‑distribution and out‑of‑distribution scenarios, achieving strong action‑label efficiency and real‑robot generalization.

By Zhaochong An, Fei Zhang, Menglin Jia, Duncan Frost, Zijian Zhou, Yikai Wang, Xudong Wang, Aditya Patel, Belinda Zeng, Tao Xiang, Serge Belongie, Amir Bar, Sen He
arXiv AI
Sep 18

Improving Cross-embodiment Transfer in Latent Action Models with Action-Similarity Supervision

The paper proposes using action‑similarity supervision to improve cross‑embodiment transfer in latent action models (LAMs). By training the similarity between latent actions to match the similarity of ground‑truth robot action sequences—without predicting the actions themselves—the authors reduce sensitivity to background noise and embodiment differences. Experiments on RoboTwin 2.0 show that this approach more than doubles cross‑embodiment success compared to predicting ground‑truth actions, especially when similarities are computed on end‑effector motion and compared across robots.

By Maxime Alvarez, Renzo Caballero, Tatsuya Matsushima, Yusuke Iwasawa, Yutaka Matsuo