← Back to all news
arXiv AI By Owen Kwon, Pablo Ortega-Kral, Arthur Bucker, Jean Oh

What Can One Sentence Fix? A Failure Taxonomy and a Minimal Language Correction for Frozen VLA Policies

Read the original on arXiv AI →

The paper investigates whether a single descriptive sentence can correct failures in frozen vision‑language‑action (VLA) policies. By labeling failures in the LIBERO‑90 benchmark, the authors identify four failure modes, finding that only execution misalignment—where the policy reaches the correct target but misjudges distance or force—can be remedied with a concise human‑generated sentence. This sentence, parsed by an LLM into a bias on the policy’s actions, successfully restores performance on both simulated and real‑world tasks, while automated language models fail to achieve similar gains.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 2

When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models

The paper investigates a specific failure mode in vision‑language‑action models, termed instruction‑action binding, where models respond to language and vision separately but fail to combine them to select the correct action under counterfactual changes. Through behavioral analyses of fine‑tuned policies, the authors show that failed rollouts often preserve source behavior or switch to other demonstrated tasks, indicating that language is not ignored but mis‑bound. They propose Equivariant Counterfactual Training (ECT), which supplies counterfactual demonstrations and a paired loss to enforce correct action selection, achieving significant performance gains across simulated and real‑world benchmarks.

By Hung-Jen Chen, Yu-Hsun Hou, Yan-Hong Chen, Yan-Fu Chen, Binghua Cai, Min Sun, Chun-Yi Lee
fine-tuningmultimodal
More like this →
arXiv Machine Learning
Aug 19

VLCP: Vision Language Control Policy Closed-Loop Code Replanning for Robot Manipulation

VLCP (Vision Language Control Policy) is a training‑free robot manipulation approach that keeps a vision‑language model (VLM) frozen and uses it to generate short Python control functions. Unlike traditional methods that retry a fixed policy, VLCP rewrites the control code every K steps based on multi‑view RGB, proprioceptive state, and state delta, allowing failures to be corrected within the same episode. In a 57‑task MuJoCo/RoboVerse benchmark, VLCP achieves 35.1% pooled success versus 3.5% for a single‑query baseline, with a 27.3% within‑episode recovery rate on failed grasps and efficient token usage.

By Dhia Naouali, Minghan Wu, Claudia Wong, Abhinav Puthran, Omar G. Younis
llmsroboticsfine-tuningmultimodal
More like this →
arXiv AI
4d ago

RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models

RePO‑VLA is a recovery‑driven policy optimization framework for Vision‑Language‑Action models that separates success, recovery, and failure trajectories. It introduces Recovery‑Aware Initialization, a Progress‑Aware Semantic Value Function, and Value‑Conditioned Refinement to learn from both successful and failed rollouts, enabling the policy to prefer high‑progress actions without online failure detectors. The authors also present the Failure‑Recovery Benchmark (FRBench) and demonstrate that RePO‑VLA raises adversarial success rates from 20% to 75% on average in simulated and real‑world bimanual tasks.

By Weijia Liufu, Xiaoyu Guo, Ruiyi Chen, Jingzhi Liu, Kaidong Zhang, Xiwen Liang, Jianqi Lin, Dawei Sun, Yuze Wang, Rongtao Xu, Bingqian Lin, Bowen Yang, Tongtong Cao, Bowen Peng, Dongyu Zhang, Guangrun Wang, Min Wang, Liang Lin, Xiaodan Liang
roboticsmultimodalbenchmarkssafety
More like this →