arXiv Computer Vision

TrapVLA: Trapping Vision-Language-Action Models in Configured Failure Modes

The paper introduces Configured Failure Trapping, a new backdoor attack targeting Vision‑Language‑Action (VLA) models that activates through subtle textual triggers and forces the robot to fail in a specific, controlled manner. It presents a data engine for generating high‑quality target trajectories, an automated evaluation suite, and two benchmarks—Trap‑LIBERO and Trap‑RoboTwin—covering four failure modes. The authors propose TrapVLA, a method that learns trigger‑induced action residuals to steer policies toward the desired failure behavior, demonstrating effectiveness in both simulation and real‑world robotic experiments while maintaining performance on clean data.

arXiv Machine Learning

Oct 12

VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models

VLA-FAIL introduces a lightweight framework for detecting failures in vision-language-action models used in robotic manipulation. It combines two novel detectors: the first-layer Mahalanobis distance (LLMD) identifies out-of-distribution states by comparing token-wise deviations in last-layer features to training data, while the second, action chunk consistency (ACC), detects failures by spotting inconsistencies between consecutive action chunks in receding-horizon control. The authors also propose a threshold‑independent metric, “AUCPDT,” to evaluate detection accuracy and latency, and show through real‑world and simulation experiments that LLMD and ACC together provide reliable and early failure detection, outperforming more expensive baseline methods.

By Florian Seligmann, Emiliyan Gospodinov, Enes Ulas Dincer, Gerhard Neumann

robotics multimodal benchmarks

arXiv AI

Sep 28

Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring

The paper introduces Hide-and-Seek, a framework for detecting failures in Vision‑Language‑Action (VLA) models during robot execution. It treats failure detection as a coarsely supervised learning problem, using inter‑trajectory and intra‑trajectory contrastive objectives to localize failure‑indicative actions without step‑level annotations. Experiments on LIBERO, VLABench, and a real‑world robotic platform show that Hide‑and‑Seek achieves state‑of‑the‑art multi‑task failure detection performance across several VLA policies.

By Seongheon Park, Wendi Li, Changdae Oh, Samuel Yeh, Zsolt Kira, Michael Hagenow, Sharon Li

robotics multimodal benchmarks

arXiv Computer Vision

Sep 7

FailureSpot: Label-Efficient Timestamp-Level Failure Detection for Vision-Language-Action Models

The paper introduces FailureSpot, a label‑efficient method for detecting failures at the timestamp level in vision‑language‑action (VLA) policies. It first generates weak supervision from unlabeled VLA action chunks by identifying abnormal patterns, then employs active learning to annotate only the most uncertain trajectories. Experiments on multiple VLA policies demonstrate improved performance for both timestamp‑level and trajectory‑level failure detection.

By Jie Ma, Zongxi Liu, Yi Zhu

robotics fine-tuning multimodal

arXiv Computer Vision

Aug 25

Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models

arXiv:2512.01946v4 Announce Type: replace-cross Abstract: Robust robotic manipulation requires reliable failure detection and recovery. Although recent Vision-Language Models (VLMs) show promise in r...

By Paul Pacaud, Ricardo Garcia, Shizhe Chen, Cordelia Schmid

llms robotics multimodal benchmarks

arXiv AI

Jun 9

ActProbe: Action-Space Probe for Early Failure Detection of Generative Robot Policies

arXiv:2606. 08508v1 Announce Type: cross Abstract: Generative robot policies fail unpredictably at deployment: they hesitate at critical moments, drift off-task, or commit to unrecoverable actions.

By Bingjia Huang, Xiangyu Li, Xiang Wang, Liang Mi, Zixu Hao, Weijun Wang, Hao Wu, Kun Li, Yunxin Liu, Ting Cao

reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jun 2

SilentDrift: Exploiting Action Chunking for Stealthy Backdoor Attacks on Vision-Language-Action Models

arXiv:2601. 14323v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models are increasingly deployed in safety-critical robotic applications, yet their security vulnerabilities remain underexplored.

By Bingxin Xu, Yuzhang Shang, Binghui Wang, Emilio Ferrara

rag robotics multimodal safety

arXiv AI

Sep 1

Training-Free Action Correction for VLA Model Failures via Language Feedback

arXiv:2608.29967v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models demonstrate strong semantic understanding yet exhibit systematic failures during deployment. The conditions under...

By Owen Kwon, Pablo Ortega-Kral, Arthur Bucker, Jean Oh

robotics multimodal

arXiv AI

Jun 9

Benchmarking Vision-Language-Action Models on SO-101: Failure and Recovery Analysis

arXiv:2606. 08881v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have demonstrated strong generalization in robotic manipulation, yet existing evaluations are primarily conducted in simulation or on expensive robotic platforms, leaving their robustness on affordable real-world robots largely unexplored.

By Yi Yu, Xinchuan Qiu

robotics fine-tuning multimodal benchmarks

arXiv AI

Aug 5

Structure-Aware Robust Fine-Tuning: Defending Vision-Language-Action Robots Against Physical Attention Hijacking

arXiv:2608. 03231v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies promise general robotic manipulation, but their robustness against physical-world attacks remains fragile.

By Jinquan Zhang, Dongfu Yin, Run Yang, Yufeng Yan, Zhen Tian, F. Richard Yu

robotics fine-tuning multimodal safety

arXiv AI

Oct 1

Learning from Runtime Feedback through Failure-Bank Self-Evolution for Vision-Language-Action Models

The paper introduces FailBank, a four‑stage self‑evolving framework that transforms runtime feedback from safety shields into lasting policy improvements for vision‑language‑action (VLA) models. By using a counterfactual correction teacher, outcome‑aware admission, and guarded LoRA updates, FailBank converts useful shield proposals into corrective targets while preserving successful actions as anchors. Experiments on the VLA‑Arena benchmark show that FailBank boosts task success rates by up to 8.5 percentage points and reduces cumulative policy cost by up to 35.6%, outperforming both base policies and traditional runtime shielding.

By Mingyue Cui, Zheyuan Liu, Yihan Zhu, Zheyuan Zhang, Meng Jiang

robotics fine-tuning multimodal benchmarks safety

arXiv Computer Vision

Sep 22

MaskVLA: Visual Masking Against Trajectory Overfitting of Vision-Language-Action Model

arXiv:2609.23565v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models integrate vision-language understanding with executable robot actions, enabling end-to-end learning for robot contr...

By Yuxuan Jiang, Jiaying Huang, Ge Wang, Shenhao Yan, Jiahao Yang, Chengsi Yao, Qi Liu, Qing Zhao, Shuguang Cui, Yiming Zhao, Yatong Han, Zhen Li

robotics fine-tuning multimodal

arXiv Computer Vision

Oct 1

Exploiting Vulnerabilities: Universal Adversarial Attacks on Vision-Language-Action Models in Robotics

arXiv:2609.39178v1 Announce Type: cross Abstract: Recently, Vision-Language-Action (VLA) models have revolutionized robotic manipulation by seamlessly integrating visual perception, language understa...

By Songhua Yang, Ziyu Liu, Yuanwei Liu, Xuetao Li, Xuanye Fei, He Huang, Zheng Wang, Miao Li

robotics multimodal safety

Related stories