← Back to all news
arXiv AI By Hayder Tirmazi

Context Compaction for World Models

Read the original on arXiv AI →

The paper "Context Compaction for World Models" introduces a benchmark to evaluate how well world models retrieve context when revisiting regions, and presents an agent harness that compacts context to extend retrieval beyond the model’s native window. It demonstrates that this compaction allows models to recall rooms up to four times farther than their default context size without retraining, while models using sliding‑window attention lose most of their retrieval capability during dreaming. The study compares four world‑model designs, showing that extending context reach via compaction benefits dreaming agents more than switching to sliding‑window attention.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 5

How To Train Your World Model: Fine-tuning vs RAG for LM-based World Modeling

The paper evaluates two strategies for training language‑model‑based world models—fine‑tuning and Retrieval Augmented Generation (RAG)—across five diverse environments. Fine‑tuning generally yields higher rewards, while RAG is more data‑efficient. The authors propose a counterfactual error estimation for RAG, improve retrieval with a hierarchical query reformulation, and combine both approaches into a hybrid system that outperforms each method alone.

By Dhananjay Ashok, Shantanu Agarwal, Vivek Datla, Jonathan May, Alfy Samuel
llmsragagentsroboticsfine-tuning
More like this →
arXiv AI
Oct 27

CTXAgent - Self-Controlled Context Management of LLM Agents

The paper introduces CTXAgent, a self‑controlled context management system for large language model agents that compacts conversation history when a token threshold is reached. It evaluates seven frontier models, comparing their ability to manage context with and without dedicated tools, and benchmarks performance across coding, scientific, and software engineering tasks. Results show that well‑managed models keep live context below 100k tokens, maintain high benchmark scores, and achieve significant cost reductions, with Claude Opus 5.5 outperforming its explicit configuration by 34% lower cost.

By Matthias Busch, Marius Tacke, Roland C. Aydin, Christian Feiler, Christian J. Cyron
llmsagentsbenchmarks
More like this →
arXiv Computer Vision
Oct 9

Learning to Retrieve: Internalizing Memory Retrieval for Video World Models

The paper introduces Learning-to-Retrieve (L2R), a method that embeds memory retrieval directly into video world models. By using the model’s internal state as a memory and employing a camera‑conditioned retrieval gate and a retrieval trigger supervised by a 3D re‑visibility signal, the approach learns when and what historical information to access. This eliminates the need for external memory systems and improves long‑term scene consistency across various base models and camera‑revisit benchmarks.

By JiaKui Hu, Tailai Chen, Yuqi Pan, Xuerui Qiu, Jialun Liu, Xiao Cao, Zhenxin Zhu, Guang Chen, Hangjun Ye, Bing Wang, Yanye Lu
benchmarks
More like this →