arXiv Computer Vision

Beyond Benchmarks: Using VLMs to Reveal Systematic Classification Failures Under Real World Conditions

The paper investigates using Vision Language Models (VLMs) to accelerate verification and validation (V&V) of classification models by automatically detecting systematic errors. It introduces a VLM-based error slice detection (ESD) method that groups and labels errors, demonstrating its ability to identify perturbations in a non-military dataset and to cluster images by surroundings in a military context. The study highlights challenges such as underrepresentation of defence data in VLM training and limited contextual diversity, and suggests that while fully automated V&V is not yet feasible, VLMs could speed up the process in the future.

arXiv AI
Aug 19

Comparative Study of Out-of-the-Box Technology for Automatic Target Detection and Recognition

The paper evaluates out‑of‑the‑box object detection models for automatic target detection and recognition (ATD/R) in military settings. Six YOLO variants and two DETR variants were benchmarked on a new military dataset featuring vehicles, occlusions, and small targets, with performance measured in mAP@0.5 and mAP@0.5:0.95 across air‑to‑ground and ground‑to‑ground perspectives. Findings show larger models and DETR-based approaches perform best, fine‑tuning on the VisDrone dataset improves air‑to‑ground and small‑object performance, yet all models still struggle with small targets in air‑to‑ground scenarios.

By Alma M. Liezenga, Lotte Nijskens, Henrik R. Baumann, Stefan Becker, Simon Bensberg, Niccol\`o Camarlinghi, H{\aa}vard R. Eiring, Alexander W. Johnsgaard, Tanel Liiv, Giuseppe Martino, Matteo Marturini, Matthias Rapp, Jan Erik van Woerden, Alexander Wolpert, Hugo J. Kuijf
arXiv AI
Sep 2

Restrict, Don't Retrain: Inference-Time VLM Guidance for Zero-Shot Aerial Segmentation

The paper proposes a method called Restrict, Don't Retrain that enhances zero-shot aerial segmentation by using inference-time guidance from a vision‑language model (VLM). It combines a frozen foundation model that labels every pixel with two VLM queries: one to select relevant classes and another to locate small objects missed by the base model. Experiments on four aerial datasets show consistent performance gains at each stage where the base model is competent.

By Teresa DiMeola, Charles Walter, Hong Xiao
arXiv AI
Aug 10

Probing Visual Concepts in Lightweight Vision-Language Models for Automated Driving

arXiv:2603. 06054v2 Announce Type: replace-cross Abstract: The use of Vision-Language Models (VLMs) in automated driving applications is becoming increasingly common, with the aim of leveraging their reasoning and generalisation capabilities to handle long-tail scenarios.

By Nikos Theodoridis, Reenu Mohandas, Ganesh Sistu, Anthony Scanlan, Ciar\'an Eising, Tim Brophy
arXiv AI
Aug 18

OODBench: Out-of-Distribution Benchmark for Large Vision-Language Models

arXiv:2602. 18094v2 Announce Type: replace-cross Abstract: Existing Visual-Language Models (VLMs) have achieved significant progress by being trained on massive-scale datasets, typically under the assumption that data are independent and identically distributed (IID).

By Ling Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen
arXiv AI
Sep 10

SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

SAFIRE is a large-scale benchmark for fire and smoke understanding in multimodal large language models (MLLMs), featuring 83,000 captioned images across 20 scenarios and 193,000 multiple-choice VQA questions derived from a 9.7K-image subset. The benchmark evaluates 10 dimensions of performance, from basic perception to higher-order reasoning, and employs a GPT‑5.4-assisted verification pipeline to ensure annotation quality. Experiments on ten open-source MLLMs (8B–38B) reveal an average accuracy of 61.9%, highlighting significant gaps in safety-critical reasoning, while fine-tuning vision encoders on just 7% of SAFIRE data boosts fire-scene classification accuracy from 20.1% to 64.5%. All resources are publicly available at https://risys-lab.github.io/SAFIRE/.

By Pengfei Li, Naufal Suryanto, Sicheng Zhang, Mohammad Alsharid, Muzammal Naseer
arXiv AI
Aug 12

Token-Based Detection of Spurious Correlations in Vision Transformers

arXiv:2509. 04009v2 Announce Type: replace-cross Abstract: Due to their powerful feature association capabilities, neural network-based computer vision models have the ability to detect and exploit unintended patterns within the data, potentially leading to correct predictions based on incorrect or unintended but statistically relevant signals.

By Solha Kang, Esla Timothy Anzaku, Wesley De Neve, Arnout Van Messem, Joris Vankerschaver, Francois Rameau, Utku Ozbulak