The paper introduces the notion of faithful latent world models, defined as an equivariance between an environment’s transition map and the learned latent dynamics. It proposes latent rollout consistency (LRC) as a metric for evaluating action‑conditioned latent predictions. Experiments show that current state‑of‑the‑art latent world models fail to faithfully model the environment’s dynamics, despite achieving task performance.
By Rodrigo Gonzalez Laiz, Steffen Schneider
The paper introduces a Latent World Model (LWM) for robot navigation that predicts action‑conditioned latent feature compatibility instead of reconstructing future observations. By exploiting the correlation between spatial proximity and latent feature similarity, the model evaluates action consequences directly in latent space and supports counterfactual training using sampled action sequences. The learned world model can supervise policy learning from unlabeled video and further improve policies via reinforcement learning entirely within the model, eliminating the need for action annotations and additional environment interaction.
By Zengmao Wang, Wei Gao, Shuhan Shen
World models enable agents to reason about future outcomes and learn policies from their knowledge of state transition, but existing approaches primarily focus on reconstructing future observations or...
The paper introduces world2vec, a method for learning mechanistic world models that capture true system dynamics without relying on reward signals. By learning latent dynamics from observations and actions, world2vec achieves accurate long‑horizon predictions with significantly fewer parameters than existing models such as DreamerV3, TD-MPC2, and LeWM. This efficiency enables state‑of‑the‑art control of continuous environments from pixel inputs and precise system identification across 71 benchmark attractors, illustrating the potential of world models for scientific discovery.
arXiv:2610.20160v1 Announce Type: new
Abstract: Learning world models across diverse domains calls for a unified control space, yet ac- tion interfaces vary across domains and action annotations are...
By Ziyu Zhu, Zhaoxu Sun, Fan Jiang, Chiyu Wang, Junfeng Ni, Mu Xu
T2-Z introduces a test‑time optimization framework that augments robotic world models with a learnable environment latent, enabling the model to adapt to unseen environmental conditions without explicit supervision of those factors. During training, the method alternates between optimizing the world‑model dynamics and the latent representation, allowing the latent to capture underlying environmental variation. At test time, a freshly initialized latent is optimized from a few interactions, reducing prediction error from 15.20 to 7.00 and increasing action success from 33.73% to 72.96% across simulated and real‑world tasks.
By Chenyuan Zhou, Yining Hong, Huang Huang, Yunfan Jiang, Wenlong Huang, David Held, Vitor Campagnolo Guizilini, Yue Wang, Li Fei-Fei, Jiajun Wu, Leonidas Guibas, Yejin Choi