← Back to all news
arXiv AI

World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models

Related stories

arXiv Machine Learning
Jul 17

Online Neural Space Time Memory for Dynamic Novel View Synthesis

arXiv:2607. 15271v1 Announce Type: cross Abstract: Online novel view synthesis from multi-view streaming videos faces a fundamental trade-off: maintaining a persistent, long-horizon memory to reconstruct temporarily occluded regions while operating under strict real-time constraints.

By Baback Elmieh, Lynn Tsai, Zeman Li, Srinivas Kaza, Tiancheng Sun, Gabor Csapo, Ali Behrouz, Yuan Deng, Stephen Lombardi, Steven M. Seitz, Xuan Luo
benchmarks
More like this →
arXiv Computer Vision
Sep 11

World in World: Explore the World with World Models

World in World introduces a training‑free, inference‑time interface that transforms diverse control signals—such as source‑video observations, target‑view projections, geometry renderings, and retrieved states—into camera‑ and time‑labelled visual states. These states are processed by a frozen causal video model’s self‑attention, enabling tasks like camera‑controlled rerendering, long‑horizon revisiting, and human‑motion transfer without additional training. The method employs a correspondence router and evidence‑wise attention to align token identities and regulate auxiliary channel contributions during a single denoising pass.

By Chenxi Song, Yanming Yang, Chi Zhang
More like this →
arXiv Computer Vision
Oct 9

Learning to Retrieve: Internalizing Memory Retrieval for Video World Models

The paper introduces Learning-to-Retrieve (L2R), a method that embeds memory retrieval directly into video world models. By using the model’s internal state as a memory and employing a camera‑conditioned retrieval gate and a retrieval trigger supervised by a 3D re‑visibility signal, the approach learns when and what historical information to access. This eliminates the need for external memory systems and improves long‑term scene consistency across various base models and camera‑revisit benchmarks.

By JiaKui Hu, Tailai Chen, Yuqi Pan, Xuerui Qiu, Jialun Liu, Xiao Cao, Zhenxin Zhu, Guang Chen, Hangjun Ye, Bing Wang, Yanye Lu
benchmarks
More like this →
arXiv Computer Vision
4d ago

WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models

arXiv:2610.19063v1 Announce Type: new Abstract: As video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requi...

By Nan Zhou, Jianxiong Gao, Qiuyu Wang, Wen Wang, Jiahao Wang, Yufeng Yuan, Shuaiting Li, Hao Ouyang, Yujun Shen, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
benchmarkssafety
More like this →
arXiv Computer Vision
Sep 30

Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory

arXiv:2604.08995v3 Announce Type: replace Abstract: With the advancement of interactive video generation, diffusion models have increasingly demonstrated their potential as world models. However, exi...

By Zile Wang, Zexiang Liu, Jiaxing Li, Kaichen Huang, Baixin Xu, Fei Kang, Mengyin An, Peiyu Wang, Biao Jiang, Yichen Wei, Yidan Xietian, Jiangbo Pei, Liang Hu, Boyi Jiang, Hua Xue, Zidong Wang, Haofeng Sun, Wei Li, Wanli Ouyang, Xianglong He, Yang Liu, Yangguang Li, Yahui Zhou
diffusionefficiency
More like this →
arXiv Computer Vision
Sep 22

WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory

WorldCrafter is a video world model that introduces a camera‑queryable implicit 3D‑aware memory to improve long‑horizon consistency and viewpoint control. The model compresses multi‑view evidence into a limited token budget shaped by the requested viewpoint, integrating historical observations via a memory encoder and pose‑conditioned readout before denoising. Experiments on static and dynamic scenes demonstrate significant gains in consistency and camera‑control accuracy while maintaining visual quality during minute‑scale exploration.

By Wangbo Yu, Kunhao Liu, Wenbo Hu, Shenghai Yuan, Chaoran Feng, Haiyang Zhou, Yukun Huang, Yiran Wang, Wang Zhao, Yingmin Luo, Ying Shan
efficiency
More like this →
arXiv Computer Vision
Aug 31

LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation

LayerRecall is a memory router for autoregressive video diffusion that selectively retrieves and injects historical key/value states into specific layers of the model, based on the current context. It addresses the problem that existing memory mechanisms expose nonlocal history but do not guarantee effective use, by recognizing that different layers prefer current, recent, or distant context. The method, combined with Cross‑Horizon Prediction Matching, achieves state‑of‑the‑art long‑range consistency on MemoBench and MovieBench while maintaining local continuity and incurring negligible inference overhead.

By Yixuan Ding, Jiahao Kong, Wei Huang, Ruijie Quan, Yi Yang
diffusion
More like this →