The paper investigates whether latent world models truly capture the underlying dynamics of their environments. It introduces the idea of faithful latent world models, defined as an equivariance between the environment’s transition map and the learned latent dynamics, and proposes latent rollout consistency (LRC) as a metric for evaluating action‑conditioned latent predictions. Experiments show that none of the state‑of‑the‑art latent world models studied actually faithfully model the environment’s dynamics, despite solving the tasks they were trained on.
arXiv:2610.20160v1 Announce Type: new
Abstract: Learning world models across diverse domains calls for a unified control space, yet ac- tion interfaces vary across domains and action annotations are...
By Ziyu Zhu, Zhaoxu Sun, Fan Jiang, Chiyu Wang, Junfeng Ni, Mu Xu
The paper introduces a Latent World Model (LWM) for robot navigation that predicts action‑conditioned latent feature compatibility instead of reconstructing future observations. By exploiting the correlation between spatial proximity and latent feature similarity, the model evaluates action consequences directly in latent space and supports counterfactual training using sampled action sequences. The learned world model can supervise policy learning from unlabeled video and further improve policies via reinforcement learning entirely within the model, eliminating the need for action annotations and additional environment interaction.
By Zengmao Wang, Wei Gao, Shuhan Shen
arXiv:2610.19683v1 Announce Type: new
Abstract: World models trained via latent dynamics offer a scalable path for embodied planning, yet existing methods suffer from representational collapse, compu...
By Yuwei Sun, Siyu Zhu
arXiv:2610.42096v1 Announce Type: cross
Abstract: World action models (WAMs) learn latent variables from observations and use these to both predict the evolution of the scene and to generate actions....
By Juntao Cheng, Jinzhou Tang, Fan Feng, Henrik I. Christensen, Yujia Zheng
T2-Z introduces a test‑time optimization framework that augments robotic world models with a learnable environment latent, enabling the model to adapt to unseen environmental conditions without explicit supervision of those factors. During training, the method alternates between optimizing the world‑model dynamics and the latent representation, allowing the latent to capture underlying environmental variation. At test time, a freshly initialized latent is optimized from a few interactions, reducing prediction error from 15.20 to 7.00 and increasing action success from 33.73% to 72.96% across simulated and real‑world tasks.
By Chenyuan Zhou, Yining Hong, Huang Huang, Yunfan Jiang, Wenlong Huang, David Held, Vitor Campagnolo Guizilini, Yue Wang, Li Fei-Fei, Jiajun Wu, Leonidas Guibas, Yejin Choi