arXiv AI

Cortex: Compact Behavior Cloning for Quake with Frozen Visual Features

arXiv:2607. 22739v1 Announce Type: cross Abstract: We study how far a deliberately simple behavioral-cloning policy can progress in a visually rich first-person game before adding reinforcement learning or explicit memory.

arXiv Machine Learning
Oct 9

WAM-Cache: Staleness-Bounded KV Reuse for Efficient World Action Models

WAM-Cache is a training‑free framework that reuses layerwise key‑value (KV) representations from a pretrained video Diffusion Transformer (DiT) across control chunks, recomputing only a sparse set of refreshed tokens. By selecting refresh tokens based on the action expert’s cross‑attention, visual latent surprise, and a strict age bound, it reduces DiT prefill FLOPs by 32‑42% on datasets such as RoboTwin 2.0 and LIBERO while maintaining action accuracy within 0.7‑1.8 percentage points of the dense baseline in simulation and 2.5 points on a real robot.

By Kai Ding, Yang He, Ruijie Quan, Yi Yang
arXiv Computer Vision
Aug 26

GlanceWAM: Sparse Test-Time Imagination for World-Action Models

GlanceWAM introduces a sparse test‑time imagination approach for world‑action models that decouples visual imagination from control. By asynchronously generating a single lookahead frame on a slow clock and decoding action chunks at a 48 ms control rate purely in latent space, it avoids latency while maintaining high success. The method achieves 72.2 % on the RoboCasa kitchen benchmark and 99.0 % on LIBERO, running 24× faster than synchronous baselines.

By Linhan Wang, Zijian An, Mingyuan Zhang, Chen Dai, Yi Xu, Can Cui, Zichong Yang, Yinlin Chen, Lifeng Zhou, Chang-Tien Lu
arXiv AI
Aug 5

ValueFormer: A Causal Transformer Value Function with Stage-Aware Labels for Semi-Autonomous Vision-Language-Action Policies

arXiv:2608. 02958v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies trained by behavior cloning fail silently: from the action stream alone, a collapsing rollout looks much like one making clean progress, because imitation supplies no notion of progress.

By Inkyu Sa, Konstantin Stulov, Rajat Bhageria
arXiv AI
Jun 2

STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language Models

arXiv:2606. 01790v1 Announce Type: cross Abstract: Vision-language-model-based graphical user interface (GUI) agents have shown broad automation capabilities, yet deployment is bottlenecked by a key-value (KV) cache that grows linearly with interaction steps.

By Yuhang Han, Wenzheng Yang, Yujie Chen, Xiangqi Jin, Yaojie Zhang, Siteng Huang, Linfeng Zhang
arXiv AI
Oct 2

Measuring the Stability Assumption Behind Action Chunking

The paper investigates how small action errors evolve when using action chunking in behavioural cloning. By injecting errors at each state and observing their growth under open‑loop (no replanning) and closed‑loop (replanning) regimes, the authors classify states as contracting, expanding, or unresolved. Across twelve manipulation tasks, they find that stable states are rare, error amplification is common, and that short‑horizon fitting can overestimate long‑horizon propagation. Predictors trained on camera and proprioceptive data can recover open‑loop stability but only partially capture closed‑loop dynamics, indicating that standard imitation learning does not reliably produce policies that contract errors when perturbed.

By Aryan Goyal
arXiv Computer Vision
4d ago

VITAL: From Visual Motion to Precise Action

arXiv:2610.20604v1 Announce Type: new Abstract: World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined in...

By Ao Liang, Youquan Liu, Mingxiao Li, Wei Chow, Lingdong Kong, Yue Liao, Dongyue Lu, Yu Yang, Wei Yin, Runnan Chen
arXiv AI
Aug 28

GameWAM: A World Action Model for Video Games

GameWAM is the first World-Action Model designed for native closed-loop gameplay and GUI control in modern video games. It jointly generates future visual observations and executable keyboard-mouse trajectories using parallel visual and action generative processes, block-causal conditioning, and flow matching. The model predicts gameplay/GUI mode at each step, handles heterogeneous native controls, and employs block-cycle control for long-horizon interaction, achieving competitive task success with fewer native actions than prior agents.

By Yuncheng Guo, Zhanqiu Zhang, Yiwen Guo, Weijia Li