arXiv Machine Learning
Jun 16
How Should World Models Be Evaluated? A Decision-Making-Centric Position
arXiv:2606. 15032v1 Announce Type: new Abstract: World models have rapidly become one of the central abstractions in modern AI.
Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators
arXiv:2607. 07196v1 Announce Type: cross Abstract: Across robotics, World Models (WMs) are increasingly used to evaluate action policies by simulating the consequences of actions in an imagined world, and returning a success or safety verdict.
Related stories
arXiv Machine Learning
Jun 30
How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position
arXiv:2606. 15032v2 Announce Type: replace Abstract: World models have become a central abstraction in modern AI.
arXiv Computer Vision
Sep 23
DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Driving
arXiv:2609.26792v1 Announce Type: cross Abstract: Faithfully evaluating end-to-end driving policies in simulation requires observations that are not merely photo-realistic, but preserve the scene fea...
arXiv Computer Vision
Sep 3
World-Coherent Decoding: Self-Verifying Test-Time Planning for World Action Models
World-Coherent Decoding (WCD) is a test-time planning framework for World Action Models (WAMs) that treats rollouts as falsifiable future–action hypotheses. At each decision step, WCD samples multiple candidates from a frozen WAM and ranks them using flow-based video surprisal for visual plausibility and action path effort for generation stability. After execution, the observed outcome audits the chosen imagination, producing a mismatch signal that trains a lightweight online predictor to improve future candidate selection, thereby enhancing reliability without updating the backbone model.
arXiv AI
Jun 30
WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL
arXiv:2602. 13977v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) promises to unlock capabilities beyond imitation learning for Vision--Language--Action (VLA) models, but its requirement for massive real-world interaction prevents direct deployment on physical robots.
arXiv Machine Learning
Jun 3
How Visible Are Silent Manipulation Failures? An Observability Study of False-Success Detection in Simulated Robot Episodes
arXiv:2606. 03134v1 Announce Type: cross Abstract: Imitation-learning policies for robot manipulation inherit the quality of the success labels attached to their training episodes, and those labels are usually produced by the robot's own success check.
arXiv AI
2d ago
When Reasoning Helps Action: Monitoring and Steering Chain-of-Thought in Vision-Language-Action Policies
arXiv:2610.00601v1 Announce Type: cross Abstract: Reasoning-enabled VLA policies expose chain-of-thought (CoT) traces that appear to explain and guide their actions, creating a potential interface fo...
arXiv AI
Sep 25
Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligence
The paper surveys 160 benchmarks from 2017‑2026 that evaluate predictive embodied intelligence, categorising them into policy suites, embodied agents, world‑model evaluation, and prediction‑to‑action bridges. It finds that most benchmarks are model‑agnostic, rarely compare Vision‑Language‑Action policies to world models, and seldom turn predictions into executed actions. The authors argue that the lack of benchmarks designed to directly test the closed‑loop advantage of world models prevents the field from answering whether such models truly improve robotic performance.
arXiv AI
Jun 10
A Practical Recipe Towards Improving Sim-and-Real Correlation for VLA Evaluation
arXiv:2606. 10366v1 Announce Type: cross Abstract: Simulation has become an essential tool for evaluating and improving vision-language-action (VLA) policies, offering scalable, reproducible, and controllable alternatives to costly real-world robot evaluation.
arXiv Computer Vision
Sep 22
DriveReferee: Geometric Safety Verdicts Need Not Be Learned for Driving World-Action Models
arXiv:2609.22762v1 Announce Type: new Abstract: Generative world-action models (WAMs) jointly generate future video and vehicle actions, while their action branches remain primarily optimized by expe...
arXiv Computer Vision
Sep 22
Human-Level Accuracy, Non-Human Strategies: Revealing Model-Human Divergence in Video Physical Reasoning
Video foundation models now match human accuracy on physical‑reasoning benchmarks, but a new distributional evaluation framework shows that their predictions diverge markedly from human judgments. On the Physion benchmark, ViT‑L models such as V‑JEPA2, VideoMAE‑v2, and DINOv2 achieve near‑human accuracy yet exhibit a 26.4% model‑human disagreement, far above the 4.8% human‑human disagreement, and lower agreement (kappa ~0.48 vs. 0.91). The divergence varies by task: models excel at geometric reasoning but lag on gravitational dynamics and causal chains, indicating they rely on statistical regularities rather than explicit forward simulation.
arXiv Computer Vision
Sep 10
No Free Checker: A Survey of Verifiers for Robot Policies
arXiv:2609.09250v1 Announce Type: cross Abstract: A verifier for robot policies reads a candidate behavior and returns a score for how well it did, used both to evaluate vision-language-action polici...