arXiv:2608.29590v1 Announce Type: new
Abstract: We propose a societal bias evaluation method for large vision-language models (LVLMs) in the era of strong safety guardrails. Existing benchmarks rely...
By Yusuke Hirota, Michael Ross Boone, Arun George Zachariah, Jibin Rajan Varghese, Yu-Chiang Frank Wang, Boyi Li, Ryo Hachiuma
The paper evaluates Vision Language Models (VLMs) on Visual Question Answering tasks where questions violate Grice's maxims. By generating question modifiers that add non-essential, ambiguous, or false information, the authors show that VLMs such as ChatGPT, Claude, Gemini, and Llava exhibit reduced performance. They also compare human pragmatic reasoning to VLM reasoning, noting differences in how each handles human‑induced versus AI‑generated violations, and find that humans spend less time resolving VLM‑induced violations while VLMs are less accurate in those cases.
By Monika Shah, Sudarshan Balaji, Somdeb Sarkhel, Sanorita Dey, Deepak Venugopal
arXiv:2609.18345v1 Announce Type: new
Abstract: Large vision-language models (LVLMs) are commonly used with only a single text prompt as the input, or plus an image. In this paper, we demonstrate tha...
By Xiaomeng Wang, Martha Larson, Zhengyu Zhao
The paper investigates whether vision‑language models (VLMs) can communicate visually in ways similar to humans. It reports that simple visual symbols, such as a red heart, can unexpectedly steer VLMs toward recursive solutions in unrelated tasks, revealing a generation‑interpretation gap where models produce intuitive shapes that do not effectively influence other models. In more complex communication games, VLMs can creatively use visuals to convey abstract concepts, though some messages remain unintuitive to humans, highlighting both the potential and the challenges of visual communication between humans and VLMs.
By Alexa R. Tartaglini, Zhengxuan Wu, Thadd\"aus Wiedemer, Martin Wattenberg, Been Kim, Yuxuan Li
Vision‑language models (VLMs) can lose accuracy when images are resized, even with minimal changes. The study shows that such small visual configuration changes—like tiling or token arrangement—cause more correctness flips across multiple checkpoints and benchmarks. Interestingly, in many cases the models still read the correct answer but fail to use it, and attention interventions reveal that configuration shifts weaken the use of readable information. By guiding models with field cues and their own transcriptions, the authors correct 97.2% of these errors.
By Dingyang Lin, Yingfeng Luo, Chenglong Wang, Chenwei Zhu, Anxiang Ma, Jingbo Zhu, Tong Xiao