arXiv Machine Learning

ActionSplice: In-Flight Action Editing for Interactive World Models

ActionSplice is an inference framework for chunk‑autoregressive video world models that allows in‑flight action editing without re‑sampling completed evaluations. It formulates the problem as Counterfactual State Transport (CST), using a lightweight corrector to move the backbone representation toward the state induced by a revised action at the same solver step. Two variants, CST*R and CST*T, update either the entire active chunk or only its suffix, achieving significant reductions in rollback‑relative LPIPS and providing speedups over waiting.

Hugging Face Trending Papers
Sep 8

Hi-FLoop: Hierarchical State-Feedback Loops for Multi-Timescale World Modeling

Hi-FLoop introduces a hierarchical state‑feedback framework for multi‑agent traffic simulation that reconciles decision time scales over an 8‑second rollout. The model uses eight scene‑level Worlds to maintain joint hypotheses, with an 8‑second Goal, 2‑second Preview, and 1‑second Control hierarchy, and commits only executed prefixes every 0.5 seconds to preserve factual consistency. A joint preview interaction graph and a prefix‑frozen A‑to‑B cascade enable sparse interaction refinement and accurate state recovery, achieving an overall score of 0.689987 on the H‑D public‑validation split and strong oracle‑minADE performance. whyItMatters":"The paper presents a novel multi‑timescale approach that improves consistency and realism in long‑horizon traffic simulations, as evidenced by its competitive evaluation metrics."

arXiv Computer Vision
Oct 28

How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop

The paper introduces a denoised counterfactual protocol to accurately measure the selectable headroom of world action models (WAMs) and uses these measurements to build a fail‑gated selector. Across four WAMs and two benchmarks, the study shows that the standard estimate overstates selectable headroom, that structured action perturbations can add up to 16 points of headroom, and that imagined futures retain candidate‑outcome information without loss. The selector improves closed‑loop task success from 76.9 % to 81.6 % on held‑out training episodes and from 62.6 % to 69.8 % on unseen tasks.

By Haojia Gao, Yue Cao, Yumin Liu, Haohua Que, Mingkai Liu, Guanqiu Guo, Zhuo Zou, Bokui Chen, Xueqian Wang
arXiv Computer Vision
Sep 4

Do Video Generators Track the World Across Segments? A Benchmark and Method for World-State Reasoning in Video Continuation

The paper introduces Statebench, a benchmark for evaluating how well video generators track world states across segments, focusing on past-visible, occluded-process, and complex-transition states. It also proposes Stateagent, a method that maintains an explicit entity-state representation, updates it with new prompts, and uses the resulting state to guide video continuation. Experiments show Stateagent raises the overall state score from 45.2 to 69.3 and improves one‑minute story generation.

By Yingmao Miao, Pengfei Zhang, Chaoran Xu, Meng Yu, Jing Tang, Xiangxiang Chu, Chao Shen, Chenhao Lin
arXiv AI
Oct 27

What Can One Sentence Fix? A Failure Taxonomy and a Minimal Language Correction for Frozen VLA Policies

The paper investigates whether a single descriptive sentence can correct failures in frozen vision‑language‑action (VLA) policies. By labeling failures in the LIBERO‑90 benchmark, the authors identify four failure modes, finding that only execution misalignment—where the policy reaches the correct target but misjudges distance or force—can be remedied with a concise human‑generated sentence. This sentence, parsed by an LLM into a bias on the policy’s actions, successfully restores performance on both simulated and real‑world tasks, while automated language models fail to achieve similar gains.

By Owen Kwon, Pablo Ortega-Kral, Arthur Bucker, Jean Oh
arXiv AI
Oct 16

ValueFormer: Stage-Aware Value Labels for Online Failure Detection and VLA Post-Training

ValueFormer introduces stage‑aware, continuous value labels for online failure detection in vision‑language‑action policies. The method uses a 3.5 M‑parameter causal transformer over a frozen DINOv3 backbone to emit per‑frame smooth Monte‑Carlo values for advantage estimation and sharp binary signals for mistake detection. In experiments, reusing teleoperation intervention flags boosts detection AP from 0.38 to 0.82 and flags 95 % of mistakes early; on a bimanual sandwich station, per‑frame post‑training weighting raises task completion from 70 % to 85 % and eliminates a repeat‑pick failure mode.

By Inkyu Sa, Konstantin Stulov, Rajat Bhageria
arXiv AI
Jul 20

Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Driving

arXiv:2607. 15621v1 Announce Type: cross Abstract: Large language models bring instruction following and scene reasoning to end-to-end driving, but their inference latency collides with the control rate a vehicle requires.

By Yun Li, Jiachen Gong, Simon Thompson, Ehsan Javanmardi, Qunli Zhang, Zifan Zeng, Shiming Liu, Peng Wang, Zixuan Guo, Manabu Tsukada
arXiv Machine Learning
Oct 2

When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models

The paper investigates a specific failure mode in vision‑language‑action models, termed instruction‑action binding, where models respond to language and vision separately but fail to combine them to select the correct action under counterfactual changes. Through behavioral analyses of fine‑tuned policies, the authors show that failed rollouts often preserve source behavior or switch to other demonstrated tasks, indicating that language is not ignored but mis‑bound. They propose Equivariant Counterfactual Training (ECT), which supplies counterfactual demonstrations and a paired loss to enforce correct action selection, achieving significant performance gains across simulated and real‑world benchmarks.

By Hung-Jen Chen, Yu-Hsun Hou, Yan-Hong Chen, Yan-Fu Chen, Binghua Cai, Min Sun, Chun-Yi Lee