← Back to all news
arXiv Computer Vision By Sahasra Kondapalli, Lara Radovanovic, Aadi Palnitkar, Mingyang Mao, Xiaomin Lin

LADBench: A Benchmark for Logical Fault Detection in Images

Read the original on arXiv Computer Vision →

LADBench is a new benchmark comprising over 1,000 synthetic images that contain logical anomalies across four domains—Residential, Urban, Collaborative, and Nature. It introduces a Tiered Prompting Protocol that progressively discloses information to test how much explicit help a model needs to locate and reason about these faults. Evaluation of leading foundation models shows that even the best only reaches 70.11% accuracy, indicating that implicit logical fault detection remains largely unsolved.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Sep 1

Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes

arXiv:2506.09557v2 Announce Type: replace-cross Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive performance in benign scenarios, their cognitive reliability deteriorat...

By Zhaoyang Wei, Bowen Jiang, Xumeng Han, Jiashu Li, Xuehui Yu, Yuling Liu, Guorong Li, Zhenjun Han, Jianbin Jiao
llmsreinforcement-learningmultimodalbenchmarkssafety
More like this →
arXiv AI
Aug 12

Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes

arXiv:2608. 10954v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive performance in benign scenarios, their cognitive reliability deteriorates significantly in complex scenes under adverse conditions.

By Zhaoyang Wei, Bowen Jiang, Xumeng Han, Jiashu Li, Xuehui Yu, Yuling Liu, Guorong Li, Zhenjun Han, Jianbin Jiao
llmsreinforcement-learningmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 2

Hob-VL: A Benchmark for Visually Grounded Boolean Reasoning

Hob‑VL is a benchmark for visually grounded Boolean reasoning that tests models on two tasks: determining whether a Boolean rule holds in an image and identifying the unique object that satisfies a Boolean description. It contains 6,000 balanced Yes/No questions built from ten visual statements across 1,000 generated scenes and 46 labeled photographs, plus 1,000 object‑identification questions on the same photographs. The questions are designed to challenge reasoning with misleading local cues, nested logical operations, and both symbolic and natural‑language presentations, revealing significant gaps in current model performance.

By Yuzhou Wang, Emile Anand, Ijay Narang
benchmarks
More like this →