The paper introduces the AGI Maze Prediction Datasets and Benchmark, a lightweight, procedurally generated grid‑world testbed for evaluating predictive models, particularly Transformers, on tasks such as per‑step transition prediction, fixed‑horizon state prediction, and sequential textual‑observation prediction. It compares byte‑level Transformer baselines with two memory‑augmented architectures, showing that a pseudo‑video spatial‑memory Transformer achieves perfect validation accuracy on selected tasks and improves sequential text‑trace prediction, while a generic auxiliary latent‑memory Transformer does not consistently help. The study highlights that structured, task‑aligned working memory can be more effective than merely increasing latent capacity, and positions the benchmark as a compact setting for testing architectures that couple textual interfaces to learned structured state.
By Alexey Potapov
arXiv:2610.19825v1 Announce Type: cross
Abstract: Integrating world models into Vision-Language-Action models (VLAs) has emerged as a promising direction, enabling policies to anticipate future dynam...
By Qisen Ma, Yixiang Chen, Yuan Xu, Peiyan Li, Jiabing Yang, Kai Wang, Jianhua Yang, Jianlou Si, Jun Huang, Jing Liu, Nianfeng Liu, Yan Huang, Liang Wang
The paper introduces the AGI Maze Prediction Datasets and Benchmark, a lightweight testbed for evaluating how Transformers and other models learn world dynamics. The benchmark, built from procedurally generated grid worlds, includes per‑step transition prediction, fixed‑horizon state prediction, and sequential textual‑observation prediction, with source‑maze‑disjoint training and validation splits to test transferable action‑conditioned dynamics. Experiments show that a pseudo‑video spatial‑memory Transformer, which initializes and updates a two‑dimensional latent workspace from the input map and action history, achieves perfect validation accuracy on selected tasks and improves sequential text‑trace prediction, outperforming byte‑level and unstructured‑memory baselines and suggesting that structured, task‑aligned working memory is more effective than additional latent capacity alone.
arXiv:2609.32193v2 Announce Type: replace
Abstract: Vision Language Action (VLA) models condition actions directly on current visual and language context, without an explicit account of how the scene...
By Hongyi Cai, Yi Herng Ong, Tingshiuan C. Wu, Chiew Hui Lim, Hanxia Li, Kehong Guo, Sze Yuan Cheong
arXiv:2506. 06006v3 Announce Type: replace-cross Abstract: Can unified vision-language models (VLMs) perform forward dynamics prediction (FDP), i.
By Yifu Qiu, Yftah Ziser, Anna Korhonen, Shay B. Cohen, Edoardo M. Ponti
arXiv:2610.20160v1 Announce Type: new
Abstract: Learning world models across diverse domains calls for a unified control space, yet ac- tion interfaces vary across domains and action annotations are...
By Ziyu Zhu, Zhaoxu Sun, Fan Jiang, Chiyu Wang, Junfeng Ni, Mu Xu
arXiv:2610.09940v1 Announce Type: cross
Abstract: Joint-embedding predictive architectures (JEPAs) predict masked or future observations in representation space, offering a natural source of predicti...
By Yuchen Zhu, Chenyi Xu, Yulin Zhang, Gang Xu, Wentao Zhu
arXiv:2608. 11605v1 Announce Type: new Abstract: World Action Models (WAMs) couple future visual prediction with robot action generation, enabling policies to model how the physical world evolves during interaction.
By Jiakai Huang, Zhongbo Wu, Zheng Zhang, Zihan Wang, Shan You, Tao Huang
arXiv:2609.34981v2 Announce Type: replace
Abstract: World action models (WAMs) predict the future alongside actions during training. Due to the heavy computation cost of video denoising, whether the...
By Renping Zhou, Zanlin Ni, Zihao Fan, Guohao Fu, Zeyu Liu, Hao Shi, Jie Zhang, Chi Bene Chen, Yang Yue, Xueyang Fu, Gao Huang
arXiv:2606. 05979v1 Announce Type: cross Abstract: We propose world-language-action (WLA) models as a new class of embodied foundation models.
By Yi Yang, Zhihong Liu, Siqi Kou, Yiyang Chen, Yanzhe Hu, Jianbo Zhou, Boyuan Zhao, Zhijie Wei, Xiao Xia, Xueqi Li, Pengfei Liu, Zhijie Deng
JEPA‑TTT is a method that continuously adapts the latent dynamics predictor of a pretrained Joint‑Embedding Predictive Architecture (JEPA) world model during test time. It performs self‑supervised updates across episodes while keeping the visual encoder and reward head fixed, using dense replay to sample prediction windows from a growing buffer. In experiments on eight dynamics shifts across four continuous‑control environments, JEPA‑TTT reduces latent prediction error by 83% and improves planning performance by 153% compared to the frozen model.
By Zheyuan Zhang, Suyu Ye, Nakul Agarwal, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Daniel Khashabi, Tianmin Shu, Vaishnav Tadiparthi
The paper introduces Dwell, a model‑based reinforcement learning framework that treats a pretrained vision‑language model (VLM) as a recurrent latent transition operator. Dwell enables the VLM to predict, adjust, and act within its own latent space, allowing the agent to learn from real interactions and latent imagination without reconstructing full futures. Experiments across five interactive visual environments show that Dwell outperforms other trainable methods while significantly reducing computational demands for imagination.
By Haoxi Li, Sikai Bai, Jianfei Ma, Qinglin Hou, Xiaocheng Lu, Tao Han, Jie Zhang, Song Guo