← Back to all news
arXiv AI By Nikolai Smolyanskiy, Jonathan Shock

Reward Observability and the Limits of Offline Checkpoint Selection in RSSM World Models

Read the original on arXiv AI →

The paper investigates the closed‑loop behavior of a recurrent state‑space model (RSSM) trained on human demonstrations in LunarLander‑v3. Using the model for zero‑shot CEM MPC and actor‑critic training in imagination, the authors find that a model‑based A2C policy trained on a single checkpoint achieves returns comparable to the best model‑free A2C policy while requiring far fewer real transitions. They introduce the Reward Observability Fraction (ROF) to quantify how much of the reward gradient lies in the observable subspace of the latent dynamics, but show that ROF does not predict the collapse of MPC in offline settings, leaving the problem of predicting closed‑loop failure unresolved.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

Hugging Face Trending Papers
Jul 2

Predicting Closed-Loop Performance of Latent World Models: Offline Checkpoint Selection for MPC and Model-Based RL Under Non-Markovian Rewards in LunarLander

We study how to predict the downstream closed-loop performance of a learned latent world model from validation-time diagnostics alone. Choosing the right checkpoint from a world-model training run is difficult: validation loss and multi-step prediction RMSE keep improving long after closed-loop performance has collapsed.

reinforcement-learning
More like this →
arXiv Machine Learning
Sep 1

The Intervention Gap in Latent World Models

The paper introduces the concept of intervention fidelity in latent world models, measuring whether a model’s open‑loop transitions align with actual environment interventions. Experiments on TD‑MPC2, Cheetah, and DreamerV3 show that high reward fit does not guarantee fidelity, and that self‑supervised models can outperform task‑anchored ones in preserving intervention effects. The authors propose a capture‑gated audit to localize failures and argue that fidelity must be directly audited on the model’s native interface.

By Donna Vakalis
robotics
More like this →
arXiv AI
Sep 25

AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control

AD-WM is a new action‑discriminative joint‑embedding world model designed for counterfactual model predictive control. It augments residual latent dynamics with action‑recovery regularization based on inverse dynamics and conditional mutual information, while discarding auxiliary heads at test time so that MPC remains unchanged. Experiments on OGBench‑Cube and other simulation environments show substantial gains in hard‑start success and mean success, and zero‑shot transfer to a Franka robot improves pick‑and‑place success from 42.2% to 71.1%.

By Jiabin Qiu, Zixuan Chen, Hongye Cao, Jieqi Shi, Jing Huo, Yang Gao
rag
More like this →
arXiv Machine Learning
Oct 9

CausalDreamer: Learning Predictive World Models with Latent Disentanglement

CausalDreamer enhances generative world models by freezing the original video tokenizer and re-encoding its latent space into a factored representation that separates controllable versus uncontrollable dynamics and reward-relevant versus reward-irrelevant information. The pretrained dynamics model is fine‑tuned to predict this factored representation, enabling more accurate model‑predictive planning. On the MMBench2 benchmark, CausalDreamer improves normalized scores by 14% on clean tasks and 25% on manipulated variants compared to the baseline, while both models perform poorly on entirely new environments.

By Prince Jha, Nils Lukas, Kun Zhang, Salem Lahlou
agentsnlpfine-tuning
More like this →