← Back to all news
arXiv AI By Kaiwen Zhu, Dongfang Liu, Liangkai Liu

VLA-Scope: Shift-Aware Failure Prediction for Vision-Language-Action Models

Read the original on arXiv AI →

VLA-Scope is a two‑stage framework designed to predict failures in vision‑language‑action models under distribution shifts. The first stage detects out‑of‑distribution inputs and classifies their shift categories using pooled image and language representations. For OOD inputs, the second stage updates failure risk during execution by combining shift category, action‑prefix features, and execution progress, achieving a ROC‑AUC of 0.8497 after 60 actions and outperforming baseline methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Sep 7

FailureSpot: Label-Efficient Timestamp-Level Failure Detection for Vision-Language-Action Models

The paper introduces FailureSpot, a label‑efficient method for detecting failures at the timestamp level in vision‑language‑action (VLA) policies. It first generates weak supervision from unlabeled VLA action chunks by identifying abnormal patterns, then employs active learning to annotate only the most uncertain trajectories. Experiments on multiple VLA policies demonstrate improved performance for both timestamp‑level and trajectory‑level failure detection.

By Jie Ma, Zongxi Liu, Yi Zhu
roboticsfine-tuningmultimodal
More like this →
arXiv AI
Sep 4

FailBench: How Reliable are VLMs at Judging Robot Task Success?

FailBench is a new benchmark for robot failure detection, containing 2,197 manipulation attempts from 14 public sources, with 75% of failures occurring naturally. The study evaluates 13 vision‑language model (VLM) detectors, finding the best model achieves only 0.77 mean balanced accuracy, and that fine‑tuned failure detectors often underperform general‑purpose VLMs. Performance varies with visual evidence, excelling when object motion is observable but dropping to near chance on contact‑intensive assembly tasks, and input‑level cropping of outcome‑relevant regions improves the top detector by 2.4 percentage points.

By Zaruhi Navasardyan, Tatul Danielyan, Hrant Davtyan
llmsroboticsfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv Machine Learning
1d ago

Beyond Appearance Shifts: Task-Semantic Action Calibration for VLA Models

The paper introduces BAS‑VLA, a task‑semantic action calibration framework for vision‑language‑action models that addresses two failure modes: unnecessary action drift under appearance changes and insufficient behavioral change under semantic alterations. BAS‑VLA uses a breaking‑centered calibration core and a selective evidence‑gated preserving auxiliary to maintain performance on clean and semantics‑preserving conditions while suppressing stale‑task behavior. Experiments on OpenPI‑pi0.5 and LIBERO‑Object Milk‑Swap show high success rates on clean and preserved tasks, a dramatic drop under target‑object swaps, and improved robustness to style shifts from 42% to 70% without harming clean performance.

By Shuaijun Liu, Feiyang You, Chengyu Wu, Shuyang Hao, Chenglong Zhang, Jingyao Cai, Xingwei Chen, Ningxin Su
roboticsmultimodalbenchmarks
More like this →