arXiv Computer Vision
Oct 21

LADBench: A Benchmark for Logical Fault Detection in Images

LADBench is a new benchmark comprising over 1,000 synthetic images that contain logical anomalies across four domains—Residential, Urban, Collaborative, and Nature. It introduces a Tiered Prompting Protocol that progressively discloses information to test how much explicit help a model needs to locate and reason about these faults. Evaluation of leading foundation models shows that even the best only reaches 70.11% accuracy, indicating that implicit logical fault detection remains largely unsolved.

By Sahasra Kondapalli, Lara Radovanovic, Aadi Palnitkar, Mingyang Mao, Xiaomin Lin
arXiv Computer Vision
4d ago

Drone-PUR: From Visual Clue Discovery to Understanding and Reasoning in UAV Scenes

Drone-PUR is a new dataset and benchmark for vision‑language models that focuses on visual clue discovery, interpretation, and integration in UAV scenes. It contains 11 tasks with 175k training and 20k test samples across single images, image sequences, cross‑modal images, and multiple viewpoints, all of which are human‑reviewed. Evaluation of 22 general‑purpose VLMs shows persistent gaps in spatial judgment and cross‑modal/cross‑view reasoning, while fine‑tuning on Drone‑PUR improves accuracy by up to 18 percentage points.

By Jiahe Wu, Bing Cao, Xingyue Guo, Ruipu Zhao, Beiming Yu, Jiekang Feng, Qinghua Hu, Pengfei Zhu
arXiv AI
Sep 1

Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes

arXiv:2506.09557v2 Announce Type: replace-cross Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive performance in benign scenarios, their cognitive reliability deteriorat...

By Zhaoyang Wei, Bowen Jiang, Xumeng Han, Jiashu Li, Xuehui Yu, Yuling Liu, Guorong Li, Zhenjun Han, Jianbin Jiao
arXiv AI
Aug 12

Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes

arXiv:2608. 10954v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive performance in benign scenarios, their cognitive reliability deteriorates significantly in complex scenes under adverse conditions.

By Zhaoyang Wei, Bowen Jiang, Xumeng Han, Jiashu Li, Xuehui Yu, Yuling Liu, Guorong Li, Zhenjun Han, Jianbin Jiao