arXiv AI By Alexandru-Bogdan Dura, Sebastian Balmus, Radu Tudor Ionescu

RoboVAD: A Large Cross-Domain Evaluation Benchmark for Anomaly Detection in Robotic Arm Manipulation Videos

Read the original on arXiv AI →

RoboVAD is a large-scale benchmark for video anomaly detection in robotic arm manipulation, featuring cross-domain evaluation where both tasks and anomaly types are unseen during training. The dataset challenges existing VAD methods, with state-of-the-art approaches, including a new method tailored for robotic arms, still achieving less than 70% micro-averaged frame-level AUC in the hardest setting. The authors provide the dataset and code publicly for further research.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Oct 12

VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models

VLA-FAIL introduces a lightweight framework for detecting failures in vision-language-action models used in robotic manipulation. It combines two novel detectors: the first-layer Mahalanobis distance (LLMD) identifies out-of-distribution states by comparing token-wise deviations in last-layer features to training data, while the second, action chunk consistency (ACC), detects failures by spotting inconsistencies between consecutive action chunks in receding-horizon control. The authors also propose a threshold‑independent metric, “AUCPDT,” to evaluate detection accuracy and latency, and show through real‑world and simulation experiments that LLMD and ACC together provide reliable and early failure detection, outperforming more expensive baseline methods.

By Florian Seligmann, Emiliyan Gospodinov, Enes Ulas Dincer, Gerhard Neumann
arXiv AI
2d ago

RoboCurate: Harnessing Diversity with Action-Verified Synthetic Data for Robot Learning

RoboCurate is a framework that generates synthetic robot data with verified action labels by executing predicted actions in a simulator and comparing the resulting motion to the generated video. It expands scene diversity using generative models with structured editing and filters out physically implausible videos with a vision‑language model before action verification. The approach yields large performance gains across multiple manipulation benchmarks, outperforming real‑data‑only baselines by up to 179.9% in certain settings.

By Seungku Kim, Suhyeok Jang, Byungjun Yoon, Dongyoung Kim, John Won, Jinwoo Shin
arXiv Computer Vision
Aug 25

Frame-Level Evaluation in Weakly Supervised Video Anomaly Detection Mostly Measures Video-Level Ranking

The paper investigates how weakly supervised video anomaly detectors, trained with only video‑level labels, are evaluated using frame‑level metrics such as Micro‑AUROC and AP. It shows that these metrics largely measure a detector’s ability to separate different videos rather than correctly ordering anomalous moments within a single video, a phenomenon termed temporal dilution. Experiments demonstrate that a detector can achieve high pooled scores even when it assigns the same score to every frame in a video, indicating that current evaluation practices may overstate temporal localization performance.

By Inpyo Song, Jangwon Lee