arXiv:2603. 16689v2 Announce Type: replace Abstract: Next-token predictors often appear to develop internal representations of the latent world and its rules.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
arXiv:2606. 28751v1 Announce Type: new Abstract: We propose a path-space formulation of prediction in AI world models.
The paper introduces the AGI Maze Prediction Datasets and Benchmark, a lightweight, procedurally generated grid‑world testbed for evaluating predictive models, particularly Transformers, on tasks such as per‑step transition prediction, fixed‑horizon state prediction, and sequential textual‑observation prediction. It compares byte‑level Transformer baselines with two memory‑augmented architectures, showing that a pseudo‑video spatial‑memory Transformer achieves perfect validation accuracy on selected tasks and improves sequential text‑trace prediction, while a generic auxiliary latent‑memory Transformer does not consistently help. The study highlights that structured, task‑aligned working memory can be more effective than merely increasing latent capacity, and positions the benchmark as a compact setting for testing architectures that couple textual interfaces to learned structured state.
arXiv:2511. 05963v4 Announce Type: replace Abstract: Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens.
The paper introduces the AGI Maze Prediction Datasets and Benchmark, a lightweight testbed for evaluating how Transformers and other models learn world dynamics. The benchmark, built from procedurally generated grid worlds, includes per‑step transition prediction, fixed‑horizon state prediction, and sequential textual‑observation prediction, with source‑maze‑disjoint training and validation splits to test transferable action‑conditioned dynamics. Experiments show that a pseudo‑video spatial‑memory Transformer, which initializes and updates a two‑dimensional latent workspace from the input map and action history, achieves perfect validation accuracy on selected tasks and improves sequential text‑trace prediction, outperforming byte‑level and unstructured‑memory baselines and suggesting that structured, task‑aligned working memory is more effective than additional latent capacity alone.
arXiv:2609.39892v1 Announce Type: new Abstract: Looped Transformers repeatedly apply the same set of Transformer layers, giving them a recurrent architecture for latent computation. Their strong perf...
arXiv:2606. 09936v1 Announce Type: cross Abstract: World models are now built on substantially different computational substrates.