arXiv Computer Vision

Human versus Computer Vision

arXiv AI
Sep 30

Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models

The study compares human and vision‑language model (VLM) responses to cross‑modal association tasks, using identical stimuli (a pseudo‑word and two images) and recording both choices and eye movements. While larger VLMs show some alignment with human choices, their attention patterns correlate poorly with human gaze, performing no better than a simple center‑bias baseline. Fine‑tuning VLMs on human choices improves choice alignment but not attention alignment, and training on human gaze improves attention correlation without affecting choice accuracy.

By Sumin Hong, Katsumi Ibaraki, Renee Shi, David Chiang, Toby Jia-Jun Li
arXiv Computer Vision
Oct 23

AVAtt: A Free-Viewing Eye-Tracking Dataset and Benchmark for Visual Attention on Paintings

AVAtt is a free‑viewing eye‑tracking dataset comprising Images{} paintings from sixteen art movements and five centuries, with Scanpaths{} scanpaths and Fixations{} fixations collected from Participants{} participants. The dataset provides individual scanpaths, fixation timing, eye‑screen distance, fixation and saliency maps, temporal saliency videos, and rich art metadata. A benchmark of classic and deep saliency models shows significant room for improvement, and 44 scanpath‑similarity measures—including a new language‑free image‑encoder family—are ranked.

By Mohamed Amine Kerkouri, Marouane Tliba, Aladine Chetouani, Alessandro Bruno
arXiv Computer Vision
Sep 1

Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration

arXiv:2608.30210v1 Announce Type: cross Abstract: AI image generators now create face portraits that are hard to tell from real photographs. Vision-language models (VLMs) are increasingly proposed to...

By Sunwhi Kim (Hwasung Medi-Science University, Dept. of Bio-Healthcare), Sunyul Kim (Yonsei University, Graduate School of Engineering, Dept. of Artificial Intelligence), Meounggun Jo (Hoseo University), Jini Tae (Gwangju Institute of Science and Technology, School of Humanities and Social Sciences)
arXiv AI
Oct 31

Telling Is Not Seeing: Textual Personas Rarely Make Vision-Language Models Answer Like the People They Describe

The paper investigates whether adding textual personas to vision‑language models (VLMs) causes the models to perceive images as the described individuals do. Experiments across 14 VLMs show that persona prompts never outperform simple rule‑based baselines and actually erase real differences between groups such as people with low vision, color blindness, or hearing loss. While the models can predict how visual impairments affect humans, the personas do not alter the models’ image representations or answer probabilities, indicating that the models merely state what they know about people rather than seeing like them.

By Jiawei Chen
arXiv AI
Oct 31

Evaluating Visual Realism of AI-Generated Videos with Eye Tracking

The paper explores using eye tracking to evaluate the visual realism of AI-generated videos, supplementing traditional explicit feedback. Two Visual Turing Test experiments with 90 participants produced 4,286 eye‑tracking trials across 438 videos, revealing that gaze patterns correlate with explicit judgments. The authors introduce GazeEval, a computational model that predicts perceived realism from eye‑tracking data with 71.3% accuracy, and they release the dataset for future research.

By Danqing Shi, Lan Jiang, Jake Lyons, Miri Zilka