arXiv Computer Vision

A VLM Answer Is Not an Anomaly Score: Rank Compression Across Image and Video Anomaly Detection

arXiv Computer Vision
Aug 24

A VLM Answer Is Not an Anomaly Score: Rank Compression in Training-Free Video Anomaly Detection

The paper investigates how vision‑language models (VLMs) can be used for training‑free video anomaly detection (VAD) by answering questions about video segments. It shows that the way a VLM’s output is converted into an anomaly score—either by taking only the most likely answer (generated readout) or by using the full probability distribution (probability readout)—has a significant impact on ranking performance. Across four large VLMs, the probability readout consistently outperforms the generated readout, achieving 5–13 point gains in AUROC or AP, because the generated readout compresses the rank order into only a few distinct scores.

By Inpyo Song, Jangwon Lee
arXiv AI
4d ago

Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection

The paper proposes a lightweight federated multiple‑instance learning (MIL) framework that trains only a compact MIL scorer across distributed clients while using a frozen vision‑language model (VLM) to verify high‑scoring video segments post‑hoc. Two VLM feedback interfaces are explored: a parsed text‑generation interface and a logit‑based interface that derives a continuous anomaly score from next‑token Yes/No probabilities. Experiments on UCF‑Crime with InternVL3.5‑2B and Qwen3‑VL‑2B‑Instruct show that the logit interface consistently improves frame‑level AUC and AP over the MIL baseline without requiring temporal post‑processing, whereas the text‑generation interface is more sensitive to prompts, parsers, and model choice.

By S\'ebastien Thuau, Amira Gran, Siba Haidar, Rachid Chelouah
arXiv AI
Sep 3

Fine-Grained Anomaly Perception in Wild UGC-Enhanced Images: A Comprehensive Dataset and Difference-Fusion Framework

The paper introduces a new task called Quality Anomaly Perception for UGC Image Enhancement (UEAP) and presents the first benchmark dataset, UEAP-4k, featuring fine‑grained annotations of anomaly categories, locations, and severity levels in real‑world user‑generated content. It proposes the Difference‑Fusion Anomaly Perception Method (DFAP‑UGC), which fuses explicit differences between enhanced images and their references using dense spatial querying, regional verification, and quality‑aware ranking to robustly identify localized anomalies. A Locality‑Aware Dynamic Task Prioritization (LADTP) training strategy is also introduced to enable efficient end‑to‑end learning without multi‑stage overhead, and experiments demonstrate that DFAP‑UGC outperforms adapted classical baselines.

By Yan Zhong, Gefei Chen, Qiufang Ma, Zhen Wang, Zhiwei Fan, Lei Shi, Tingting Jiang
arXiv Computer Vision
Aug 25

Frame-Level Evaluation in Weakly Supervised Video Anomaly Detection Mostly Measures Video-Level Ranking

The paper investigates how weakly supervised video anomaly detectors, trained with only video‑level labels, are evaluated using frame‑level metrics such as Micro‑AUROC and AP. It shows that these metrics largely measure a detector’s ability to separate different videos rather than correctly ordering anomalous moments within a single video, a phenomenon termed temporal dilution. Experiments demonstrate that a detector can achieve high pooled scores even when it assigns the same score to every frame in a video, indicating that current evaluation practices may overstate temporal localization performance.

By Inpyo Song, Jangwon Lee
arXiv AI
Jun 26

MMGist: A Comprehensive Multimodal Benchmark for 2027

arXiv:2606. 22437v2 Announce Type: replace-cross Abstract: We conduct a systematic study of 18 widely used vision-language benchmarks and identify three major issues: 1) many items do not rely on visual cues and therefore fail to effectively measure multimodal understanding; 2) many items are already close to performance saturation for current LVLMs, which limits their discriminative power; 3) a small number of anomalous items affect the reliability of evaluation results.

By Wenzhen Yuan, Jiacheng Ruan, Wutao Xiong, Chengping Zhao, Ting Liu, Yuzhuo Fu
arXiv AI
Aug 25

What's the Catch? Evaluating Temporal Consistency in Vision-Language Models

The paper introduces TimeCatch, a benchmark that evaluates temporal consistency in vision‑language models (VLMs) by treating temporal grounding as an anomaly detection problem. Temporal anomalies are created by swapping consecutive frames, while frame‑level anomalies involve replacing a frame with Gaussian noise. Across synthetic and real‑world datasets, VLMs reliably detect and localize frame‑level anomalies but perform near chance on temporal anomaly detection, whereas humans excel at both tasks.

By Marek Hradil, Danae S\'anchez Villegas