The paper investigates whether latent world models truly capture the underlying dynamics of their environments. It introduces the idea of faithful latent world models, defined as an equivariance between the environment’s transition map and the learned latent dynamics, and proposes latent rollout consistency (LRC) as a metric for evaluating action‑conditioned latent predictions. Experiments show that none of the state‑of‑the‑art latent world models studied actually faithfully model the environment’s dynamics, despite solving the tasks they were trained on.
The paper presents a reward‑free continual learning framework for space robots that uses latent‑state world models to adapt to severe hardware degradation. By pre‑training a model‑based agent in diverse simulations, the world model learns to predict reward structure in latent space. During deployment, the observation encoder and reward predictor are frozen while only the transition dynamics are updated via unsupervised rollouts, allowing the policy to adapt using imagined trajectories without new rewards.
By Andrej Orsula, Miguel Olivares-Mendez, Carol Martinez
The paper introduces the notion of faithful latent world models, defined as an equivariance between an environment’s transition map and the learned latent dynamics. It proposes latent rollout consistency (LRC) as a metric for evaluating action‑conditioned latent predictions. Experiments show that current state‑of‑the‑art latent world models fail to faithfully model the environment’s dynamics, despite achieving task performance.
By Rodrigo Gonzalez Laiz, Steffen Schneider
arXiv:2608.23526v1 Announce Type: new
Abstract: World models can predict video without learning dynamics that they reliably preserve. We test whether a frozen DreamerV3 trained only on pendulum video...
By Richard Bao
arXiv:2606. 20104v1 Announce Type: cross Abstract: Perception for action suggests that representations of the world should be shaped not by visual fidelity alone, but by their relevance for actions.
By Petr Ivashkov, Randall Balestriero, Bernhard Sch\"olkopf
arXiv:2611.07245v1 Announce Type: cross
Abstract: We study whether \emph{long-horizon model-based value expansion} can serve as an effective framework in pixel-based offline reinforcement learning (R...
By Kwanyoung Park, Sergey Levine