arXiv:2608. 06544v1 Announce Type: new Abstract: World models for visual control typically learn compact latent states by reconstructing observations, implicitly encouraging representations to preserve information across the entire visual input.
By SM Mazharul Islam, Manfred Huber
arXiv:2610.30724v2 Announce Type: replace-cross
Abstract: Video provides a natural source of supervision: what happens next serves as a learning signal for visual world modeling. Yet masked video pre...
By Yifeng Yao, Jinghan Li, Zhihao Zhang, Yajiao Xiong, Yang Jin, Kun Xu, Huishuai Zhang, Dongyan Zhao
Contrastive World Models propose a new method for learning latent dynamics without pixel reconstruction. By replacing observation reconstruction with a Deep InfoMax-like objective that maximizes mutual information between state-action sequences and local patch features of future observations, the approach encourages state representations to retain predictive information while ignoring visually irrelevant details. Experiments show that this method matches existing baselines in simple settings and significantly outperforms them when distractors or natural video backgrounds are present, while also training more efficiently by eliminating the pixel decoder.
By Bonnie Li
World in World introduces a training‑free inference interface that lets users control autoregressive video world models from new viewpoints. By converting diverse control signals—source‑video observations, target‑view projections, geometry renderings, and retrieved states—into camera‑ and time‑labelled visual tokens, the system uses a frozen causal video model’s self‑attention to maintain synchronization, complete unseen regions, and recover past appearances. The method supports camera‑controlled rerendering, long‑horizon revisiting, and human‑motion transfer while preserving perceptual quality, temporal consistency, and camera‑following accuracy.
World in World introduces a training‑free, inference‑time interface that transforms diverse control signals—such as source‑video observations, target‑view projections, geometry renderings, and retrieved states—into camera‑ and time‑labelled visual states. These states are processed by a frozen causal video model’s self‑attention, enabling tasks like camera‑controlled rerendering, long‑horizon revisiting, and human‑motion transfer without additional training. The method employs a correspondence router and evidence‑wise attention to align token identities and regulate auxiliary channel contributions during a single denoising pass.
By Chenxi Song, Yanming Yang, Chi Zhang
The paper introduces VANE, a self‑supervised visual pretraining framework that learns from next‑frame prediction in embedding space. It uses a causal encoder to summarize past observations, a state predictor to infer the representation of the entire next frame, and a lightweight generative decoder to reconstruct fine‑grained spatial detail. By applying the same objective to both videos and still images through a local‑to‑global crop sequence, VANE achieves state‑of‑the‑art performance on anticipation, recognition, and dense perception benchmarks, notably outperforming V‑JEPA on EPIC‑KITCHENS‑100 action anticipation.
By Yifeng Yao, Jinghan Li, Zhihao Zhang, Yajiao Xiong, Yang Jin, Kun Xu, Huishuai Zhang, Dongyan Zhao
The paper introduces a decoupled memory control framework for spatially consistent long‑horizon video generation. By separating memory conditioning from generation, it uses a lightweight memory branch to learn spatial consistency from past observations and a per‑frame cross‑attention mechanism to condition each frame on the most relevant historical data. A camera‑aware gating mechanism further ensures memory is used only when meaningful references exist, resulting in lower training costs, higher spatial consistency, and state‑of‑the‑art visual quality.
By Yanjun Guo, Zhengqiang Zhang, Pengfei Wang, Xinyue Liang, Zhiyuan Ma, Lei Zhang
arXiv:2605.15195v2 Announce Type: replace
Abstract: Recent feed-forward reconstruction models, such as VGGT, have proven competitive with traditional optimization-based reconstructors while also prov...
By Jianyuan Wang, Minghao Chen, Shangzhan Zhang, Nikita Karaev, Johannes Sch\"onberger, Patrick Labatut, Piotr Bojanowski, David Novotny, Andrea Vedaldi, Christian Rupprecht
arXiv:2611.18324v1 Announce Type: new
Abstract: Weakly Supervised Video Semantic Segmentation (WS-VSS) aims to learn temporally consistent semantic object masks in videos using only video-level label...
By Andrea Marelli, Alessandro Ardenghi, Giacomo Boracchi, Luca Magri
arXiv:2605.15618v2 Announce Type: replace-cross
Abstract: Self-supervised video models are increasingly framed as world models, yet they are still evaluated almost entirely on clean video and reporte...
By Ali J Alrasheed, Aryan Yazdan Parast, Basim Azam, James Bailey, Naveed Akhtar
The paper introduces Learning-to-Retrieve (L2R), a method that embeds memory retrieval directly into video world models. By using the model’s internal state as a memory and employing a camera‑conditioned retrieval gate and a retrieval trigger supervised by a 3D re‑visibility signal, the approach learns when and what historical information to access. This eliminates the need for external memory systems and improves long‑term scene consistency across various base models and camera‑revisit benchmarks.
By JiaKui Hu, Tailai Chen, Yuqi Pan, Xuerui Qiu, Jialun Liu, Xiao Cao, Zhenxin Zhu, Guang Chen, Hangjun Ye, Bing Wang, Yanye Lu
arXiv:2606. 07687v1 Announce Type: cross Abstract: Video world models are increasingly used to provide predictive visual representations, yet it remains unclear which pretraining signals induce action-relevant structure in their latent spaces.
By Jewon Yeom, Hanseul Kim, Jeongjae Park, Sungmok Jung, Jaejin Lee, Taesup Kim