arXiv AI

Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination

arXiv Machine Learning
Jun 9

C$^3$ache: Accelerating World Action Models with Cross Inference Chunk Cache

arXiv:2606. 08962v1 Announce Type: new Abstract: World Action Models (WAMs) generalize better than standard Vision-Language-Action (VLA) policies to novel motions and environments, because a video-modeling objective lets them learn from abundant unlabeled video rather than scarce labeled robot demonstrations.

By Weisen Zhao, Lam Nguyen, Zhicong Lu, Yuzhang Shang
arXiv Computer Vision
Oct 16

Complementary Future Token Pruning for World Action Models

The paper introduces CF‑Prune, a training‑free method that prunes future‑frame tokens in pretrained World Action Models (WAMs) to reduce inference latency. By using instruction and action attention during the initial denoising pass, the framework identifies task‑relevant tokens, while an entropy‑based phase identification keeps dense computation in precision phases to avoid performance loss. Experiments on RoboLab, RoboCasa, and real‑world manipulation tasks show consistent speedups—up to 1.33× at 50% pruning—while maintaining task success rates within 1 percentage point of the baseline.

By Baekseung Kim, Junhoo Lee, Sohyeon Hong, Suhyun Jeon, Minkyu Kim, Seungyeon Kim, Donghoon Han, Nojun Kwak
Hugging Face Trending Papers
6d ago

Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control

Elastic-WAM introduces an adaptive approach to world-action model refinement, allocating computational effort across time and space based on a learned spatiotemporal demand field. This field determines when to stop refinement, how far to advance each denoising step, and which future regions to refresh, allowing the model to tailor refinement depth to specific tasks and states. Experiments on simulation and real‑world long‑horizon manipulation tasks show that Elastic‑WAM achieves state‑of‑the‑art control performance while reducing inference latency.

arXiv Computer Vision
6d ago

Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control

Elastic-WAM introduces an adaptive spatiotemporal refinement strategy for world-action models, allocating computational effort across time and space based on a learned refinement demand field. This field determines when to stop refinement, how far to advance denoising steps, and which future regions to refresh, enabling efficient recurrent sparse refinement. Experiments on simulation and real‑world long‑horizon manipulation tasks show state‑of‑the‑art control performance with lower inference latency.

By Haitian Li, Tianfu Li, Xuxi Zhao, Zizheng Liu, Yangcheng Yu, Yiwen Hou, Zeyu Xiao, Mutian Xu, Ziqian Yang, Haonan Chen, Lin Shao
Hugging Face Trending Papers
Aug 6

Evidence-Driven Dynamic Visual Selector for Efficient Long Video Understanding

Recent advancements in MLLM-based long-form video understanding have mitigated inference-time computational cost and limited context lengths by selecting query-relevant frames. However, existing approaches predominantly rely on external proxy scorers and rigid heuristic rules, inevitably suffering from misalignment with the target MLLM's intrinsic evidence and failing to accommodate the non-uniform spatiotemporal information density.

arXiv Computer Vision
Aug 26

GlanceWAM: Sparse Test-Time Imagination for World-Action Models

GlanceWAM introduces a sparse test‑time imagination approach for world‑action models that decouples visual imagination from control. By asynchronously generating a single lookahead frame on a slow clock and decoding action chunks at a 48 ms control rate purely in latent space, it avoids latency while maintaining high success. The method achieves 72.2 % on the RoboCasa kitchen benchmark and 99.0 % on LIBERO, running 24× faster than synchronous baselines.

By Linhan Wang, Zijian An, Mingyuan Zhang, Chen Dai, Yi Xu, Can Cui, Zichong Yang, Yinlin Chen, Lifeng Zhou, Chang-Tien Lu
arXiv Computer Vision
Oct 9

SV-TAD: Native Sparse Convs for Efficient Temporal Action Detection

SV-TAD introduces native sparse 2D convolutions to enable convolutional adapters to operate directly on dynamically pruned token sets, addressing the inefficiency of dense reconstruction in token‑selection methods. By integrating this primitive into a temporal action detection framework, the approach cuts VideoMAEv2‑L computation by up to 64% and achieves 2.2× faster inference while preserving state‑of‑the‑art accuracy on THUMOS‑14 and ActivityNet‑1.3. When applied to InternVideoNext‑L, it surpasses prior state‑of‑the‑art performance at roughly half the computational cost and naturally supports auxiliary task tokens for improved fine‑grained assembly detection on ATTACH.

By Ricardo Pizarro, Roberto Valle, Jos\'e M. Buenaposada, Luis M. Bergasa, Luis Baumela
arXiv Computer Vision
Oct 5

Native Action-Prior Learning from Videos for World Action Models

The paper introduces NAVA-WAM, a method that learns action priors directly from observation-only videos to pretrain action policies for robots. It uses a two-stage training process: first, it pretrains on videos with future‑video flow‑matching supervision to learn action‑relevant priors, and second, it fine‑tunes with action‑labeled demonstrations for robot control. Experiments show that NAVA‑WAM outperforms previous methods in both in‑distribution and out‑of‑distribution scenarios, achieving strong action‑label efficiency and real‑robot generalization.

By Zhaochong An, Fei Zhang, Menglin Jia, Duncan Frost, Zijian Zhou, Yikai Wang, Xudong Wang, Aditya Patel, Belinda Zeng, Tao Xiang, Serge Belongie, Amir Bar, Sen He
arXiv Computer Vision
Oct 8

Video Prediction Policy 2: Predict Better, Act Better

Video Prediction Policy 2 (VPP2) is a new world action model that improves zero‑shot generalization for both video prediction and action generation. It achieves this by pretraining a large, diverse manipulation video dataset with event‑level supervision, then distilling the model into a single‑step visual planner and adding a mixture‑of‑transformers action module. Experiments show VPP2 outperforms leading baselines on open‑ended video prediction, real‑world zero‑shot manipulation, and several challenging benchmarks.

By Yanjiang Guo, Haodong Yan, Zhide Zhong, Zhongru Zhang, Qingyuan Yang, Qingzhou Lu, Xiaoyu Chen, Yen-Jen Wang, Shuying Deng, Chenghan Yang, Puzhen Yuan, Chenxin Liu, Tun Ban, Xiang Zhu, Yichen Liu, Kun Feng, Haoang Li, Jianyu Chen