arXiv Computer Vision

Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control

Elastic-WAM introduces an adaptive spatiotemporal refinement strategy for world-action models, allocating computational effort across time and space based on a learned refinement demand field. This field determines when to stop refinement, how far to advance denoising steps, and which future regions to refresh, enabling efficient recurrent sparse refinement. Experiments on simulation and real‑world long‑horizon manipulation tasks show state‑of‑the‑art control performance with lower inference latency.

Hugging Face Trending Papers
6d ago

Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control

Elastic-WAM introduces an adaptive approach to world-action model refinement, allocating computational effort across time and space based on a learned spatiotemporal demand field. This field determines when to stop refinement, how far to advance each denoising step, and which future regions to refresh, allowing the model to tailor refinement depth to specific tasks and states. Experiments on simulation and real‑world long‑horizon manipulation tasks show that Elastic‑WAM achieves state‑of‑the‑art control performance while reducing inference latency.

arXiv Computer Vision
Oct 23

Self-Distilling Future Video Representations for World Action Models

The paper introduces FRED, a framework that self‑distills future video representations for World Action Models (WAMs) by predicting these representations from noise using ground‑truth future observations as targets. By conditioning on historical observations, FRED improves the quality of future representations obtained in a single denoising step, thereby enhancing action prediction efficiency. Experiments on real‑world manipulation tasks show that FRED outperforms existing WAMs, especially on long‑horizon, out‑of‑domain, and zero‑shot scenarios, achieving a 50.0% success rate on tasks only seen in human videos versus 19.4% for the best competitor.

By Huiwon Jang, Suhyeok Jang, Hyeonmo Kang, Seungku Kim, Jinwoo Shin, Sihyun Yu
arXiv AI
Jun 9

AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing

arXiv:2606. 09811v1 Announce Type: cross Abstract: World-action models have emerged as a promising paradigm for robot manipulation, jointly modeling visual scene dynamics and actions to inject physical priors into policy learning.

By Jisong Cai, Long Ling, Shiwei Chu, Zhongshan Liu, Jiayue Kang, Zhixuan Liang, Wenjie Xu, Yinan Mao, Weinan Zhang, Xiaokang Yang, Ru Ying, Ran Zheng, Yao Mu
arXiv AI
Sep 25

Rolling-WAM: World Action Models with Rolling Imagination

Rolling-WAM is a new formulation for World Action Models that spreads the joint video-action denoising process across multiple replanning cycles. It keeps a sliding window of video-action chunks at different noise levels, fully denoising the immediate chunk for execution while partially refining future chunks. This approach reduces latency, improves closed-loop responsiveness, and achieves a 4.5× speedup in steady-state replanning compared to standard WAMs while maintaining competitive manipulation performance.

By Yinghua Zhou, Junjie Ye, Yiqi Zhao, Hao Dong, Celina Shiyu Wang, Ruohai Ge, Tingyi Yang, Basile Van Hoorick, Gaurav Sukhatme, Vitor Guizilini, Yue Wang
arXiv Computer Vision
Sep 3

Spatially Aware World Action Model via Geometric Latent Diffusion

The paper introduces Spatially Aware World Action Model (SA‑WAM), a diffusion‑based framework that extends existing World Action Models by incorporating depth information alongside RGB to enable 3‑D‑aware action and future‑state prediction. SA‑WAM repurposes a pretrained video diffusion model, using a nonlinear encoding to map unbounded depth into the tokenizer’s bounded domain, thus preserving pretrained visual priors without 3‑D‑specific fine‑tuning. The model achieves state‑of‑the‑art performance on RoboCasa and LIBERO‑Plus benchmarks and demonstrates superior real‑world performance on a UR5 robotic arm in randomized environments, while also providing analysis linking world‑model prediction quality to rollout success.

By Javier Alejandro Lopetegui Gonzalez, Paul Pacaud, Cordelia Schmid
arXiv Machine Learning
Jun 9

C$^3$ache: Accelerating World Action Models with Cross Inference Chunk Cache

arXiv:2606. 08962v1 Announce Type: new Abstract: World Action Models (WAMs) generalize better than standard Vision-Language-Action (VLA) policies to novel motions and environments, because a video-modeling objective lets them learn from abundant unlabeled video rather than scarce labeled robot demonstrations.

By Weisen Zhao, Lam Nguyen, Zhicong Lu, Yuzhang Shang
arXiv Computer Vision
Sep 16

World-Action Models for Robot Learning and Control: A Survey

The survey "World-Action Models for Robot Learning and Control" reviews recent advances in coupling future world prediction with executable action generation for robots in open environments. It clarifies the scope of World-Action Models (WAMs) relative to conventional world models, model-based RL, and Vision‑Language‑Action policies, and organizes existing methods through a unified taxonomy covering representations, transition modeling, action interfaces, architectures, training pipelines, data modalities, and scaling strategies. The paper also surveys applications in manipulation, navigation, and autonomous driving, summarizes datasets, benchmarks, and metrics, and discusses key challenges such as action alignment, spatial consistency, long‑horizon memory, and efficient inference.

By Zuxing Lu, Hongjia Zhai, Guanzhi Wang, Huajian Zeng, Jiaqi Yang, Jingyu Liu, Lei Cheng, Yuantai Zhang, Yuheng Qiu, Zezhou Cheng, Ivan Laptev, Danfei Xu, Benjamin Riviere, Giuseppe Loianno, Eric Xing, Xingxing Zuo