arXiv AI
When Does Test-Time Search Help World Action Models? An Empirical Audit of Candidate Selection and Simulator Integrity
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computer Vision
Oct 28
How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop
The paper introduces a denoised counterfactual protocol to accurately measure the selectable headroom of world action models (WAMs) and uses these measurements to build a fail‑gated selector. Across four WAMs and two benchmarks, the study shows that the standard estimate overstates selectable headroom, that structured action perturbations can add up to 16 points of headroom, and that imagined futures retain candidate‑outcome information without loss. The selector improves closed‑loop task success from 76.9 % to 81.6 % on held‑out training episodes and from 62.6 % to 69.8 % on unseen tasks.
arXiv AI
Sep 15
Sampling headroom is not selection gain: a compute-value audit of test-time scaling for video world models
The paper introduces the Compute-Value Audit (CVA), a sequential framework that evaluates whether extra sampling during test‑time scaling for video world models actually yields a net benefit after accounting for the compute cost of generation and verification. On 192 Physics‑IQ scenes, increasing the sample pool from 4 to 16 candidates improves oracle quality by +9.23 IQ, yet common metrics such as Flow, Cycle, and VideoReward fail to reliably recover this headroom, and adaptive‑depth policies recover only 42‑69% of the potential gain. Only a few specific interventions—anchor‑explorer in a sparse PRM800K setting, MMLU‑Pro exposing a predictive‑state gap, and a privileged paired‑future upper bound—successfully pass all CVA stages, indicating that sampling headroom is valuable only when it can be converted into a reliable decision that survives the full compute charge.
arXiv AI
Jul 24
Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models
arXiv:2607. 20436v1 Announce Type: cross Abstract: Safety evaluations often assume that behavior observed during testing reflects behavior in ordinary use, but fine-tuning can break this assumption.
arXiv AI
Jun 16
Auditing Reward Hackability in Code RL Training Environments
arXiv:2606. 16062v1 Announce Type: new Abstract: We measure the rate at which code RL environments accept incorrect solutions as correct.
arXiv AI
Jul 14
Interaction Scaling: Grounding the Third Axis of Test-Time Compute
arXiv:2607. 11598v1 Announce Type: new Abstract: There are two standard ways to spend more compute at test time: let a model reason longer, or sample more attempts and keep one.
arXiv AI
Aug 6
The Order Is the Guarantee: Verifier-Budgeted Code Deletion with Static-First Learned Proposals
arXiv:2608. 04611v1 Announce Type: cross Abstract: Frontier coding models now match or exceed strong human reference points on programming benchmarks, yet benchmark success does not imply maintainable software.