arXiv Computer Vision October 7, 2026 By Ruotian Zhang, Xiaoxiao Sun, Junzhe Huang, James Burgess, Serena Yeung-Levy

SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images

Read the original on arXiv Computer Vision →

SalArt-VQA is a diagnostic benchmark that tests whether vision‑language models can understand salient artifacts in AI‑generated images. It includes 950 images and 3,681 multiple‑choice questions that assess artifact presence, semantic localization, spatial grounding, and evidence‑grounded defect identification. The benchmark reveals that high image‑level detection accuracy can mask failures in grounded understanding, showing a trade‑off between sensitivity and calibration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Aug 10

SABRE: Scalable and Automated Benchmarking of VLMs under Stress

arXiv:2608. 07435v1 Announce Type: cross Abstract: Vision-language models (VLMs) are improving rapidly, but benchmark development lags behind, making weaknesses hard to identify.

By Zixuan Lan, Luzhe Sun, Matthew R. Walter, Jiawei Zhou

llms multimodal benchmarks

arXiv AI

Jun 16

Tool-IQA: Augmenting Image Quality Assessment with Simple Tools

arXiv:2606. 16082v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have been increasingly adopted for Image Quality Assessment (IQA).

By Guanyi Qin, Junjie Zhang, Chunming He, Yibing Fu, Jie Liang, Tianhe Wu, Lei Zhang

llms agents multimodal benchmarks

arXiv Computer Vision

Aug 24

AGIDefect-4K: A Richly Annotated Dataset for AI-Generated Image Defect Detection, Localization and Explanation

arXiv:2608.20713v1 Announce Type: new Abstract: Generative AI can now produce highly realistic images, yet current models still exhibit subtle but critical defects that undermine their reliability. W...

By Xiangfei Sheng, Weidong Zou, Tianjiao Gu, Zhichao Yang, Pengfei Chen, Leida Li

llms computer-vision multimodal benchmarks

arXiv Computer Vision

Sep 15

Beyond OCR Accuracy: Text-Centric VQA Under Image Degradation with Modular and End-to-End

arXiv:2609.13815v1 Announce Type: new Abstract: Text-centric Visual Question Answering (VQA) requires reading and reasoning over text embedded in images, a task made substantially harder when images...

By Ritali Vatsi, Rachapudi Jagadeesh, Shruti Singh Baghel, Himani Sharma, Amit Shukla, Pawan Goyal

llms computer-vision nlp fine-tuning multimodal

arXiv AI

Jul 21

Seeing What Is Actually There: PriVE-Bench and PriVE-Tools for Counterfactual Evaluation of Agentic Visual Evidence in VLMs

arXiv:2607. 16311v1 Announce Type: cross Abstract: Vision-language models (VLMs) often answer visual questions using learned language and category priors rather than grounding their predictions in the image itself.

By Jingyu Sun, Jiachen Tu, Yuyang Xue, Yaoxin Jiang, Guoyi Xu, Zhengtao Yao, Rui Qian, Yizheng Sun, Hongpeng Zhou, Jingyuan Sun, Yan Lin

llms agents multimodal benchmarks

arXiv AI

Sep 10

VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification

arXiv:2609.06245v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) perform strongly on general visual understanding tasks such as visual question answering, yet they often str...

By Yixin Wan, Tianle Zheng, Kai-Wei Chang

llms computer-vision nlp multimodal benchmarks

One email a morning, machine-written

Related stories

SABRE: Scalable and Automated Benchmarking of VLMs under Stress

Tool-IQA: Augmenting Image Quality Assessment with Simple Tools

AGIDefect-4K: A Richly Annotated Dataset for AI-Generated Image Defect Detection, Localization and Explanation

Beyond OCR Accuracy: Text-Centric VQA Under Image Degradation with Modular and End-to-End

Seeing What Is Actually There: PriVE-Bench and PriVE-Tools for Counterfactual Evaluation of Agentic Visual Evidence in VLMs

VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification