arXiv Computer Vision
Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Machine Learning
3d ago
Self-Refining Vision Language Model for Robotic Failure Detection and Reasoning
The paper introduces ARMOR, an Adaptive Round-based Multi-task Model designed for robotic failure detection and reasoning. It treats detection and reasoning as a self‑refinement process, iteratively predicting outcomes and natural language explanations conditioned on prior outputs. Training leverages both large sparse binary labels and small rich reasoning annotations through offline and online imitation learning, while inference uses multiple refinement trajectories and a self‑certainty metric to select the best prediction.
arXiv AI
Sep 4
FailBench: How Reliable are VLMs at Judging Robot Task Success?
FailBench is a new benchmark for robot failure detection, containing 2,197 manipulation attempts from 14 public sources, with 75% of failures occurring naturally. The study evaluates 13 vision‑language model (VLM) detectors, finding the best model achieves only 0.77 mean balanced accuracy, and that fine‑tuned failure detectors often underperform general‑purpose VLMs. Performance varies with visual evidence, excelling when object motion is observable but dropping to near chance on contact‑intensive assembly tasks, and input‑level cropping of outcome‑relevant regions improves the top detector by 2.4 percentage points.
arXiv AI
Jun 30
RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis
arXiv:2606. 28385v1 Announce Type: cross Abstract: Recent advances in robot world models enable synthetic video generation for embodied prediction and planning.
arXiv Machine Learning
Oct 12
VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models
VLA-FAIL introduces a lightweight framework for detecting failures in vision-language-action models used in robotic manipulation. It combines two novel detectors: the first-layer Mahalanobis distance (LLMD) identifies out-of-distribution states by comparing token-wise deviations in last-layer features to training data, while the second, action chunk consistency (ACC), detects failures by spotting inconsistencies between consecutive action chunks in receding-horizon control. The authors also propose a threshold‑independent metric, “AUCPDT,” to evaluate detection accuracy and latency, and show through real‑world and simulation experiments that LLMD and ACC together provide reliable and early failure detection, outperforming more expensive baseline methods.
arXiv Computer Vision
Aug 28
TrapVLA: Trapping Vision-Language-Action Models in Configured Failure Modes
The paper introduces Configured Failure Trapping, a new backdoor attack targeting Vision‑Language‑Action (VLA) models that activates through subtle textual triggers and forces the robot to fail in a specific, controlled manner. It presents a data engine for generating high‑quality target trajectories, an automated evaluation suite, and two benchmarks—Trap‑LIBERO and Trap‑RoboTwin—covering four failure modes. The authors propose TrapVLA, a method that learns trigger‑induced action residuals to steer policies toward the desired failure behavior, demonstrating effectiveness in both simulation and real‑world robotic experiments while maintaining performance on clean data.
Hugging Face Trending Papers
Jul 30
RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents
Vision-Language-Action (VLA) models have attracted growing interest as a scalable approach to robotic manipulation. While these models are effective action predictors, deploying them as robotic agents exposes critical gaps: no mechanism for failure recovery, inconsistent execution over long horizons, and limited robustness to shifts in observations, tasks, or embodiments.