The paper introduces a denoised counterfactual protocol to accurately measure the selectable headroom of world action models (WAMs) and uses these measurements to build a fail‑gated selector. Across four WAMs and two benchmarks, the study shows that the standard estimate overstates selectable headroom, that structured action perturbations can add up to 16 points of headroom, and that imagined futures retain candidate‑outcome information without loss. The selector improves closed‑loop task success from 76.9 % to 81.6 % on held‑out training episodes and from 62.6 % to 69.8 % on unseen tasks.
arXiv:2607. 23602v1 Announce Type: cross Abstract: Controllers based on sampling and latent world models assign a predicted terminal cost to each candidate action sequence, choose the minimum, execute its first action block, and replan.
arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.
AD-WM is a new action‑discriminative joint‑embedding world model designed for counterfactual model predictive control. It augments residual latent dynamics with action‑recovery regularization based on inverse dynamics and conditional mutual information, while discarding auxiliary heads at test time so that MPC remains unchanged. Experiments on OGBench‑Cube and other simulation environments show substantial gains in hard‑start success and mean success, and zero‑shot transfer to a Franka robot improves pick‑and‑place success from 42.2% to 71.1%.
The paper investigates a specific failure mode in vision‑language‑action models, termed instruction‑action binding, where models respond to language and vision separately but fail to combine them to select the correct action under counterfactual changes. Through behavioral analyses of fine‑tuned policies, the authors show that failed rollouts often preserve source behavior or switch to other demonstrated tasks, indicating that language is not ignored but mis‑bound. They propose Equivariant Counterfactual Training (ECT), which supplies counterfactual demonstrations and a paired loss to enforce correct action selection, achieving significant performance gains across simulated and real‑world benchmarks.
The paper introduces loss‑conditioned state execution, a model‑agnostic technique that decides whether to apply a world model’s proposed state change or keep the current state based on whether the change reduces downstream loss. It formalizes state movability as the existence of a loss‑reducing feasible correction and constructs loss‑specific proposals from predictive distributions, executing them only when a groupwise lower confidence bound on loss improvement is positive. Experiments on forecasting and dynamics benchmarks show that the method accepts updates for a subset of cases, achieving lower bounded loss than persistence or always executing the proposal, and highlights that event predictability and loss‑based decisions must be evaluated separately.
The paper introduces WebMRE, an offline benchmark comprising 541 tasks and 5,293 steps extracted from WebArena trajectories, designed to provide deterministic scoring for web agents without live environments. It enables the first systematic study of how guide sentences and grounded actions reinforce each other, showing that jointly decoding a guide improves element selection accuracy and that the guide acts as a causal instruction channel. The authors fine‑tune models that outperform leading zero‑shot baselines on all offline metrics.
The paper evaluates how well typed decision models, which output probability distributions, actually provide reliable guarantees. By comparing open-weight Laya and Jev 1.13 across 190 evaluations, the study finds that point‑estimate thresholds frequently break promised accuracy, while a Beta lower‑bound rule and Learn‑then‑Test approach maintain guarantees more consistently. The work also explores calibration across languages, the impact of noise, data requirements for fine‑tuning, and the limitations of confidence in predicting outcomes or providing faithful explanations.
The paper introduces the concept of intervention fidelity in latent world models, measuring whether a model’s open‑loop transitions align with actual environment interventions. Experiments on TD‑MPC2, Cheetah, and DreamerV3 show that high reward fit does not guarantee fidelity, and that self‑supervised models can outperform task‑anchored ones in preserving intervention effects. The authors propose a capture‑gated audit to localize failures and argue that fidelity must be directly audited on the model’s native interface.
The paper treats human interventions during LLM agent failures as labels that pinpoint the internal computation that went wrong, rather than merely training data. In a controlled environment, the authors analyze one failure type—over‑asking a clarifying question when the request is already clear—using rule‑based overseer labels validated by blind human judgment. They find that the decision to ask versus act can be causally localized across several models, that over‑asking exhibits distinct mechanistic signatures, and that steering or weight consolidation can mitigate the issue, though outcomes vary by model family.
DreamAvoid introduces a test‑time dreaming framework for Vision‑Language‑Action models to anticipate and avoid failures during critical manipulation phases. It uses a Dream Trigger to detect critical phases, samples candidate action chunks via an Action Proposer, and evaluates short‑horizon futures with a Dream Evaluator trained on success, failure, and boundary data. Experiments on real‑world and simulated tasks show DreamAvoid improves task success rates, achieving 72.5% success versus 48.8% for the base policy and 54.4% for GPC‑RANK.
The paper investigates why world models used for planning can misjudge returns despite producing visually accurate rollouts. By replacing imagined transitions with real ones at key event steps—where rewards are earned or episodes end—the authors show that correcting these few transitions eliminates about 70% of return errors, outperforming corrections based on high visual error metrics. Across 44 Atari-100k model–collection pairs, event-based patches significantly improve plan ranking, revealing a structured pattern of remaining errors that can guide further repairs.
arXiv:2610.18614v1 Announce Type: new Abstract: Controlling an acting LLM agent means deciding, at each step of a long-horizon task, whether to trust its next action, abstain, or hand off to a human....