The paper investigates whether latent world models truly capture the underlying dynamics of their environments. It introduces the idea of faithful latent world models, defined as an equivariance between the environment’s transition map and the learned latent dynamics, and proposes latent rollout consistency (LRC) as a metric for evaluating action‑conditioned latent predictions. Experiments show that none of the state‑of‑the‑art latent world models studied actually faithfully model the environment’s dynamics, despite solving the tasks they were trained on.
arXiv:2610.20160v1 Announce Type: new
Abstract: Learning world models across diverse domains calls for a unified control space, yet ac- tion interfaces vary across domains and action annotations are...
By Ziyu Zhu, Zhaoxu Sun, Fan Jiang, Chiyu Wang, Junfeng Ni, Mu Xu
The paper introduces a Latent World Model (LWM) for robot navigation that predicts action‑conditioned latent feature compatibility instead of reconstructing future observations. By exploiting the correlation between spatial proximity and latent feature similarity, the model evaluates action consequences directly in latent space and supports counterfactual training using sampled action sequences. The learned world model can supervise policy learning from unlabeled video and further improve policies via reinforcement learning entirely within the model, eliminating the need for action annotations and additional environment interaction.
By Zengmao Wang, Wei Gao, Shuhan Shen
arXiv:2610.19683v1 Announce Type: new
Abstract: World models trained via latent dynamics offer a scalable path for embodied planning, yet existing methods suffer from representational collapse, compu...
By Yuwei Sun, Siyu Zhu
arXiv:2610.42096v1 Announce Type: cross
Abstract: World action models (WAMs) learn latent variables from observations and use these to both predict the evolution of the scene and to generate actions....
By Juntao Cheng, Jinzhou Tang, Fan Feng, Henrik I. Christensen, Yujia Zheng
T2-Z introduces a test‑time optimization framework that augments robotic world models with a learnable environment latent, enabling the model to adapt to unseen environmental conditions without explicit supervision of those factors. During training, the method alternates between optimizing the world‑model dynamics and the latent representation, allowing the latent to capture underlying environmental variation. At test time, a freshly initialized latent is optimized from a few interactions, reducing prediction error from 15.20 to 7.00 and increasing action success from 33.73% to 72.96% across simulated and real‑world tasks.
By Chenyuan Zhou, Yining Hong, Huang Huang, Yunfan Jiang, Wenlong Huang, David Held, Vitor Campagnolo Guizilini, Yue Wang, Li Fei-Fei, Jiajun Wu, Leonidas Guibas, Yejin Choi
The paper introduces an auxiliary loss that aligns latent world‑model training with task success criteria by regressing success‑criterion quantities during training. This approach improves success rates on PushT and cube tasks by 3.5% and 3.4% respectively, while leaving the model’s architecture and inputs unchanged at test time.
By Takumi Hara, Kanata Suzuki
World models enable agents to reason about future outcomes and learn policies from their knowledge of state transition, but existing approaches primarily focus on reconstructing future observations or...
The paper introduces world2vec, a method for learning mechanistic world models that capture true system dynamics without relying on reward signals. By learning latent dynamics from observations and actions, world2vec achieves accurate long‑horizon predictions with significantly fewer parameters than existing models such as DreamerV3, TD-MPC2, and LeWM. This efficiency enables state‑of‑the‑art control of continuous environments from pixel inputs and precise system identification across 71 benchmark attractors, illustrating the potential of world models for scientific discovery.
arXiv:2608.24855v1 Announce Type: new
Abstract: Latent world models are inherently strong encoders that transform image pixel to latent embedding, yet existing world models still rely on online traje...
By Hsiang-Wei Huang, Jianxu Shangguan, Junbin Lu, Jenq-Neng Hwang
The paper introduces AnyGoal, a benchmark that tests world models on goal‑shifted task variants where state and action factors are recombined beyond the combinations seen in expert rollouts. It evaluates how well different world‑model families generalize to these unseen recombinations across multiple environments, revealing consistent failures. The study analyzes dynamics and representation shortcomings to identify where current models succeed and where they struggle in goal‑shifted prediction and planning.
By Heejeong Nam, Divyat Mahajan, Quentin Le Lidec, Randall Balestriero, David Klindt
arXiv:2610.28483v1 Announce Type: new
Abstract: Learning latent dynamics is fundamental to building action-conditioned world models for planning. Among existing families, Joint Embedding Predictive A...
By Yilei Man, Yang Chen, Jingcai Guo