arXiv AI August 14, 2026 By Paul Osemudiame Oamen, Owusu-Banahene Osei, Ananya Mukherjee, Christian Greisinger, Steffen Eger, Pius Onobhayedo, Wei Zhao

How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures

Read the original on arXiv AI →

arXiv:2608. 13267v1 Announce Type: cross Abstract: Existing vision-language model (VLM) benchmarks emphasize perception and reasoning accuracy (how well VLMs describe and reason about what they see in an image), with limited attention to behavioral reliability under uncertainty (how they behave when visual evidence is missing or misleading).

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Aug 10

SABRE: Scalable and Automated Benchmarking of VLMs under Stress

arXiv:2608. 07435v1 Announce Type: cross Abstract: Vision-language models (VLMs) are improving rapidly, but benchmark development lags behind, making weaknesses hard to identify.

By Zixuan Lan, Luzhe Sun, Matthew R. Walter, Jiawei Zhou

llms multimodal benchmarks

arXiv AI

Jul 17

Just Keep Prompting: Evaluating Repetitive Socratic Prompting in VLMs

arXiv:2607. 14099v1 Announce Type: cross Abstract: Deploying Vision-Language Models (VLMs) in real-world settings requires not only strong visual reasoning but also stability under sustained conversational pressure.

By Shayda Moezzi, Bishoy Galoaa, Lorena Genua, Taskin Padir, Sarah Ostadabbas

llms nlp multimodal benchmarks safety

arXiv AI

Jul 9

HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models

arXiv:2607. 07507v1 Announce Type: cross Abstract: Hallucinations in vision language models (VLMs) are commonly treated as semantic errors, yet they often arise from partial or ambiguous visual evidence.

By Feng He, Zhenting Wang, Qifan Wang, Qiang Guan, Dongfang Liu, Ruixiang Tang, Qiankun Li

llms multimodal safety

arXiv Machine Learning

Jul 27

Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation

arXiv:2607. 22034v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly deployed on consumer hardware where input images are degraded by compression, camera shake, and poor lighting.

By M M Asif Ferdous

llms multimodal

arXiv AI

Aug 6

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models

arXiv:2604. 14888v3 Announce Type: replace-cross Abstract: Recent advances in vision language models (VLMs) offer reasoning capabilities, yet how these unfold and integrate visual and textual information remains unclear.

By Danae S\'anchez Villegas, Samuel Lewis-Lim, Nikolaos Aletras, Desmond Elliott

llms multimodal safety

arXiv AI

Jul 2

MindEdit-Bench: Benchmarking Object-Level Counterfactual Spatial Reasoning in VLMs from In-the-Wild Photos

arXiv:2607. 00491v1 Announce Type: cross Abstract: Benchmarks for vision-language models (VLMs) mostly test observational spatial reasoning: models describe relations already visible in the input.

By Leyuan Yu, Xiao Tang, Minghao Liu, Xinyuan Li, Xiaokai Bai, Sheng Zhou, Qunshu Lin, Weihao Xuan, Naoto Yokoya

llms multimodal benchmarks

Related stories

SABRE: Scalable and Automated Benchmarking of VLMs under Stress

Just Keep Prompting: Evaluating Repetitive Socratic Prompting in VLMs

HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models

Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models

MindEdit-Bench: Benchmarking Object-Level Counterfactual Spatial Reasoning in VLMs from In-the-Wild Photos