arXiv Machine Learning
Aug 10
Addressable Memory for Video World Models
arXiv:2608. 07408v1 Announce Type: cross Abstract: We study visual persistence in interactive video world models.
GROME: GROunding Factorized MEmory into Video World Models
GROME introduces a factorized memory framework for video world models that separates historical evidence into static and dynamic components. It uses local memory retrieval to align static history spatially and injects the factorized memory into models without extra training. Experiments show improved long‑horizon scene persistence and dynamic entity fidelity while maintaining visual quality and controllability.
Related stories
arXiv Computer Vision
Oct 9
Learning to Retrieve: Internalizing Memory Retrieval for Video World Models
The paper introduces Learning-to-Retrieve (L2R), a method that embeds memory retrieval directly into video world models. By using the model’s internal state as a memory and employing a camera‑conditioned retrieval gate and a retrieval trigger supervised by a 3D re‑visibility signal, the approach learns when and what historical information to access. This eliminates the need for external memory systems and improves long‑term scene consistency across various base models and camera‑revisit benchmarks.
arXiv Computer Vision
Oct 28
Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generation
The paper introduces a decoupled memory control framework for spatially consistent long‑horizon video generation. By separating memory conditioning from generation, it uses a lightweight memory branch to learn spatial consistency from past observations and a per‑frame cross‑attention mechanism to condition each frame on the most relevant historical data. A camera‑aware gating mechanism further ensures memory is used only when meaningful references exist, resulting in lower training costs, higher spatial consistency, and state‑of‑the‑art visual quality.
arXiv Computer Vision
Aug 31
LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation
LayerRecall is a memory router for autoregressive video diffusion that selectively retrieves and injects historical key/value states into specific layers of the model, based on the current context. It addresses the problem that existing memory mechanisms expose nonlocal history but do not guarantee effective use, by recognizing that different layers prefer current, recent, or distant context. The method, combined with Cross‑Horizon Prediction Matching, achieves state‑of‑the‑art long‑range consistency on MemoBench and MovieBench while maintaining local continuity and incurring negligible inference overhead.
arXiv Computer Vision
Sep 24
The Past Frames the Future: Memory for Autoregressive Video Generation
arXiv:2609.28466v1 Announce Type: new Abstract: Advances in generative models have improved video fidelity, enabling long-horizon generation, interactive world modeling, and evolving visual environme...
arXiv Computer Vision
Sep 18
Can 4D Foundation Models Remember?
The paper "Can 4D Foundation Models Remember?" introduces PersistBench, a dataset and metric suite that uses 360° videos to evaluate visual memory in 4D foundation models. It focuses on three aspects—object permanence, motion continuity, and appearance preservation—to assess how well models remember objects after they leave the field of view. Experiments show that current models only maintain short‑term consistency, revealing a significant gap between perception and robust memory.
arXiv AI
Oct 15
World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
arXiv:2610.17434v1 Announce Type: cross Abstract: Recent advances in video world models have enabled plausible interactive environments, yet long-horizon generation places a fundamental demand on vis...
arXiv AI
Aug 14
AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)
arXiv:2608. 13492v1 Announce Type: new Abstract: This report presents an improved version of AlayaWorld.
arXiv Computer Vision
Oct 1
MEMO: Multi-Level Entity-Aware Memory for Streaming Video Understanding
arXiv:2609.38900v1 Announce Type: new Abstract: Streaming video understanding requires models to process unbounded visual streams while preserving rich visual semantics across vast temporal horizons,...
arXiv Computer Vision
Oct 15
WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
arXiv:2610.19063v1 Announce Type: new Abstract: As video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requi...
arXiv Machine Learning
Jun 9
Echo-Memory: A Controlled Study of Memory in Action World Models
arXiv:2606. 09803v1 Announce Type: cross Abstract: We present \textbf{Echo-Memory}, a controlled study of memory mechanisms in action-conditioned world models.
arXiv Computer Vision
Aug 28
RECAP-Forcing: Retaining Content Appearances for Long Video Generation
RECAP-Forcing is a new method for long autoregressive video generation that addresses the memory challenge by organizing memory based on appearance novelty rather than recency. The approach retains key-value caches for newly appearing content—such as entering subjects, disoccluded regions, and new scenes—at the moment they first appear, ensuring consistent identities over time. It combines an attention sink for the initial scene with an optical-flow-based novelty bank for later frames, improving visual quality and semantic fidelity without adding learnable parameters.