arXiv AI
Sep 30

Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models

The study compares human and vision‑language model (VLM) responses to cross‑modal association tasks, using identical stimuli (a pseudo‑word and two images) and recording both choices and eye movements. While larger VLMs show some alignment with human choices, their attention patterns correlate poorly with human gaze, performing no better than a simple center‑bias baseline. Fine‑tuning VLMs on human choices improves choice alignment but not attention alignment, and training on human gaze improves attention correlation without affecting choice accuracy.

By Sumin Hong, Katsumi Ibaraki, Renee Shi, David Chiang, Toby Jia-Jun Li
arXiv Computer Vision
Oct 23

AVAtt: A Free-Viewing Eye-Tracking Dataset and Benchmark for Visual Attention on Paintings

AVAtt is a free‑viewing eye‑tracking dataset comprising Images{} paintings from sixteen art movements and five centuries, with Scanpaths{} scanpaths and Fixations{} fixations collected from Participants{} participants. The dataset provides individual scanpaths, fixation timing, eye‑screen distance, fixation and saliency maps, temporal saliency videos, and rich art metadata. A benchmark of classic and deep saliency models shows significant room for improvement, and 44 scanpath‑similarity measures—including a new language‑free image‑encoder family—are ranked.

By Mohamed Amine Kerkouri, Marouane Tliba, Aladine Chetouani, Alessandro Bruno
arXiv Computer Vision
Sep 1

Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration

arXiv:2608.30210v1 Announce Type: cross Abstract: AI image generators now create face portraits that are hard to tell from real photographs. Vision-language models (VLMs) are increasingly proposed to...

By Sunwhi Kim (Hwasung Medi-Science University, Dept. of Bio-Healthcare), Sunyul Kim (Yonsei University, Graduate School of Engineering, Dept. of Artificial Intelligence), Meounggun Jo (Hoseo University), Jini Tae (Gwangju Institute of Science and Technology, School of Humanities and Social Sciences)