arXiv Machine Learning

Predictive Statistics Shape Emergent World Representations of Grid Walkers

arXiv:2603. 16689v2 Announce Type: replace Abstract: Next-token predictors often appear to develop internal representations of the latent world and its rules.

arXiv Machine Learning

Jun 30

A Path-Space Formulation of Prediction in World Models: From a Single Action to Prediction, Planning, and Irreversibility

arXiv:2606. 28751v1 Announce Type: new Abstract: We propose a path-space formulation of prediction in AI world models.

By Gunn Kim

arXiv Machine Learning

Jun 16

Next-Latent Prediction Transformers Learn Compact World Models

arXiv:2511. 05963v4 Announce Type: replace Abstract: Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens.

By Jayden Teoh, Manan Tomar, Kwangjun Ahn, Edward S. Hu, Tim Pearce, Pratyusha Sharma, Akshay Krishnamurthy, Riashat Islam, Alex Lamb, John Langford

llms diffusion benchmarks safety

arXiv AI

Jun 10

One Lens, Many Worlds : A Capability-Typed Interface for World-Model Interpretability

arXiv:2606. 09936v1 Announce Type: cross Abstract: World models are now built on substantially different computational substrates.

By Bhavith Chandra Challagundla, Sanskar Pandey, Param Thakkar, Rishikesh Mallagundla, Yugandhar Reddy Gogireddy, Wenhao Lu, Hindol Roy Choudhury, Shravani Challagundla, Mohamed Deraz Nasr, Spursh Deshpande

llms rag diffusion safety

arXiv Machine Learning

1d ago

SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry

arXiv:2608. 16287v1 Announce Type: new Abstract: Joint-embedding predictive world models plan by scoring predicted terminal embeddings against a goal embedding using a cost defined on the representation itself.

By Jiaming Hu, Yan Zheng, Tian Wang

rag safety

arXiv AI

2d ago

Traj-LeWM: Path-Aware World-Model Planning via Latent Trajectory Cost

arXiv:2608. 14125v1 Announce Type: new Abstract: LeWM is a lightweight visual world model that learns latent dynamics end-to-end from pixels and ranks candidate action sequences by the distance between their predicted endpoints and the goal.

By Xiaodi Huang, Ziyi Ding, Jingtian Wan, Yuchen Liu, Yuan Zhang, Xiao-Ping Zhang, Jiayu Chen, Zhang Zhang, Tao Huang

arXiv Machine Learning

Aug 10

Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction

arXiv:2608. 07420v1 Announce Type: new Abstract: World models are expected to support imagination over extended temporal horizons, yet most are still trained through local few-step prediction objectives and deployed by recursively rolling out their own predictions.

By Xinyi Li, Zaishuo Xia, Chenjie Hao, Yubei Chen

rag benchmarks

arXiv Machine Learning

Jun 18

Decomposing Prediction Mechanisms for In-Context Recall

arXiv:2507. 01414v2 Announce Type: replace Abstract: We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall.

By Sultan Daniels, Dylan Davis, Dhruv Gautam, Wentinn Liao, Gireeja Ranade, Anant Sahai

llms efficiency

arXiv Machine Learning

Jun 9

Trajectory Geometry of Transformer Representations Across Layers

arXiv:2606. 09287v1 Announce Type: new Abstract: Understanding how transformer representations evolve across layers, not merely what they encode, remains an open problem in mechanistic interpretability.

By Vishal Pandey, Gopal Singh

llms rag safety

arXiv AI

Jul 15

Higher Embedding Dimension Creates a Stronger World Model for a Simple Sorting Task

arXiv:2510. 18315v2 Announce Type: replace-cross Abstract: We investigate how embedding dimension affects the emergence of an internal "world model" in a transformer trained with reinforcement learning to perform bubble-sort-style adjacent swaps.

By Brady Bhalla, Honglu Fan, Nancy Chen, Tony Yue YU

llms rag reinforcement-learning safety

arXiv Machine Learning

Jun 26

Fast LeWorldModel

arXiv:2606. 26217v1 Announce Type: new Abstract: Joint-Embedding Predictive Architectures (JEPAs), including recent LeWorldModel (LeWM), have become a promising foundation for reconstruction-free visual world models.

By Yuntian Gao, Xiangyu Xu

rag

arXiv AI

Jun 2

TERRA: Task-Embedded Reasoning and Representation Architecture for Cross-Domain Applications

arXiv:2606. 01520v1 Announce Type: new Abstract: A single action-conditioned latent predictive architecture can in principle be trained on the structured state of a driving scene, a robot workspace, or a financial order book.

By Shayan Shokri

rag nlp reinforcement-learning robotics

arXiv Machine Learning

Jun 26

Learning Long-Range Dependencies with Temporal Predictive Coding

arXiv:2602. 18131v2 Announce Type: replace Abstract: Temporal Predictive Coding provides a layer-local, parallelisable mechanism for learning in recurrent systems, making it an attractive candidate for online local learning on neuromorphic and edge hardware.

By Tom Potter, Oliver Rhodes

benchmarks

Related stories