The paper investigates whether latent world models truly capture the underlying dynamics of their environments. It introduces the idea of faithful latent world models, defined as an equivariance between the environment’s transition map and the learned latent dynamics, and proposes latent rollout consistency (LRC) as a metric for evaluating action‑conditioned latent predictions. Experiments show that none of the state‑of‑the‑art latent world models studied actually faithfully model the environment’s dynamics, despite solving the tasks they were trained on.
The paper introduces the notion of faithful latent world models, defined as an equivariance between an environment’s transition map and the learned latent dynamics. It proposes latent rollout consistency (LRC) as a metric for evaluating action‑conditioned latent predictions. Experiments show that current state‑of‑the‑art latent world models fail to faithfully model the environment’s dynamics, despite achieving task performance.
The paper introduces a Latent World Model (LWM) for robot navigation that predicts action‑conditioned latent feature compatibility instead of reconstructing future observations. By exploiting the correlation between spatial proximity and latent feature similarity, the model evaluates action consequences directly in latent space and supports counterfactual training using sampled action sequences. The learned world model can supervise policy learning from unlabeled video and further improve policies via reinforcement learning entirely within the model, eliminating the need for action annotations and additional environment interaction.
World models enable agents to reason about future outcomes and learn policies from their knowledge of state transition, but existing approaches primarily focus on reconstructing future observations or...
The paper introduces world2vec, a method for learning mechanistic world models that capture true system dynamics without relying on reward signals. By learning latent dynamics from observations and actions, world2vec achieves accurate long‑horizon predictions with significantly fewer parameters than existing models such as DreamerV3, TD-MPC2, and LeWM. This efficiency enables state‑of‑the‑art control of continuous environments from pixel inputs and precise system identification across 71 benchmark attractors, illustrating the potential of world models for scientific discovery.
arXiv:2610.20160v1 Announce Type: new Abstract: Learning world models across diverse domains calls for a unified control space, yet ac- tion interfaces vary across domains and action annotations are...
T2-Z introduces a test‑time optimization framework that augments robotic world models with a learnable environment latent, enabling the model to adapt to unseen environmental conditions without explicit supervision of those factors. During training, the method alternates between optimizing the world‑model dynamics and the latent representation, allowing the latent to capture underlying environmental variation. At test time, a freshly initialized latent is optimized from a few interactions, reducing prediction error from 15.20 to 7.00 and increasing action success from 33.73% to 72.96% across simulated and real‑world tasks.
arXiv:2608.23526v1 Announce Type: new Abstract: World models can predict video without learning dynamics that they reliably preserve. We test whether a frozen DreamerV3 trained only on pendulum video...
arXiv:2606.28712v2 Announce Type: replace-cross Abstract: Classical simultaneous localization and mapping (SLAM) estimates metric poses and a geometric map but does not provide an action-conditioned...
The paper introduces AnyGoal, a benchmark that tests world models on goal‑shifted task variants where state and action factors are recombined beyond the combinations seen in expert rollouts. It evaluates how well different world‑model families generalize to these unseen recombinations across multiple environments, revealing consistent failures. The study analyzes dynamics and representation shortcomings to identify where current models succeed and where they struggle in goal‑shifted prediction and planning.
arXiv:2610.19683v1 Announce Type: new Abstract: World models trained via latent dynamics offer a scalable path for embodied planning, yet existing methods suffer from representational collapse, compu...
arXiv:2606. 28712v1 Announce Type: cross Abstract: Classical SLAM estimates metric poses and a geometric map but produces no actionable predictive model for planning.
The paper introduces Retrospective World Modeling, a new paradigm for vision‑language‑model (VLM) agents that allows them to reason backward by estimating which action most likely caused a state transition. It proposes the Self‑Consistency Reward (SCR), an intrinsic signal that measures how well a policy action aligns with this retrospective explanation, providing dense transition‑level feedback. Experiments demonstrate that incorporating SCR improves policy robustness and generalization compared to purely prospective world‑modeling approaches.