arXiv AI
Latent World Action Model for GUI Agents
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
3d ago
GUIFlow: Temporal Redundancy-Aware Incremental Inference for GUI Agents
GUIFlow is a framework that enables temporal redundancy‑aware incremental inference for GUI agents built on vision‑language models. By selectively updating computational states—such as position‑aligned KV states, vision encoder outputs, and language model drafts—across interaction steps, it allows cross‑step reuse of visual encoding, prefill, and decoding without extra training or model changes. Experiments show up to 3.39× speedup on offline benchmarks and 3.49× in online OSWorld interactions while preserving task performance.
arXiv Machine Learning
Sep 22
Contrastive World Models
Contrastive World Models propose a new method for learning latent dynamics without pixel reconstruction. By replacing observation reconstruction with a Deep InfoMax-like objective that maximizes mutual information between state-action sequences and local patch features of future observations, the approach encourages state representations to retain predictive information while ignoring visually irrelevant details. Experiments show that this method matches existing baselines in simple settings and significantly outperforms them when distractors or natural video backgrounds are present, while also training more efficiently by eliminating the pixel decoder.
arXiv AI
Jul 28
Scaling GUI Agents with Visual State Transitions
arXiv:2607. 24112v1 Announce Type: new Abstract: We introduce State Transition Pretraining (STP) as a new scaling axis for GUI agents.
arXiv Computer Vision
Oct 21
PLaCA: Learning Patch-Level Future Latent Change for Action Generation
arXiv:2610.27657v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models generate robot actions from visual observations and language instructions, but effective manipulation also requir...
arXiv AI
Sep 30
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
arXiv:2609.37250v1 Announce Type: cross Abstract: World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrai...
arXiv AI
Aug 13
Foresight Without Seeing: Latent Futures for World Action Models
arXiv:2608. 11605v1 Announce Type: new Abstract: World Action Models (WAMs) couple future visual prediction with robot action generation, enabling policies to model how the physical world evolves during interaction.