The paper introduces the concept of intervention fidelity in latent world models, measuring whether a model’s open‑loop transitions align with actual environment interventions. Experiments on TD‑MPC2, Cheetah, and DreamerV3 show that high reward fit does not guarantee fidelity, and that self‑supervised models can outperform task‑anchored ones in preserving intervention effects. The authors propose a capture‑gated audit to localize failures and argue that fidelity must be directly audited on the model’s native interface.
By Donna Vakalis
arXiv:2608. 19760v1 Announce Type: cross Abstract: Audited against causal ground truth from executed replay in a single-agent tool environment (ALFWorld), none of the step-level credit signals used to train LLM agents -- LLM-judge scores, outcome-conditioned logprob ratios, or the policy's own confidence -- identifies which steps causally matter better than chance.
By Haiyue Zhang
The paper investigates the closed‑loop behavior of a recurrent state‑space model (RSSM) trained on human demonstrations in LunarLander‑v3. Using the model for zero‑shot CEM MPC and actor‑critic training in imagination, the authors find that a model‑based A2C policy trained on a single checkpoint achieves returns comparable to the best model‑free A2C policy while requiring far fewer real transitions. They introduce the Reward Observability Fraction (ROF) to quantify how much of the reward gradient lies in the observable subspace of the latent dynamics, but show that ROF does not predict the collapse of MPC in offline settings, leaving the problem of predicting closed‑loop failure unresolved.
By Nikolai Smolyanskiy, Jonathan Shock
The paper investigates why reinforcement learning with verifiable rewards (RLVR) reduces the diversity of solutions in reasoning tasks. By analyzing the Countdown task, the authors show that RLVR contracts the solution space mainly at the entrance—before the first arithmetic operation—causing a 67% drop in solution coverage. They demonstrate that providing an unselected entrance prefix or applying entrance‑targeted interventions can restore or even improve coverage without harming accuracy.
By Qiancheng Zhou, Ruizhe Li
arXiv:2607. 20543v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) can improve one-sample accuracy while making a model worse under repeated sampling.
By Todd Zhou
arXiv:2607. 01736v1 Announce Type: cross Abstract: We study how to predict the downstream closed-loop performance of a learned latent world model from validation-time diagnostics alone.
By Nikolai Smolyanskiy