arXiv AI

Leveraging Psychophysical Attentional Distribution for Gaze-Augmented Reward Modeling

The paper proposes a new reward modeling approach that uses a spatially graded distribution of visual attention centered on the current gaze location, rather than just fixation points. By incorporating this gaze distribution into reinforcement learning from human feedback, the authors achieve higher preference prediction accuracy compared to a baseline. The results also qualitatively align with known patterns of human attention during reading, indicating that the model captures cognitively meaningful aspects of attention allocation.

arXiv AI
Sep 30

Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models

The study compares human and vision‑language model (VLM) responses to cross‑modal association tasks, using identical stimuli (a pseudo‑word and two images) and recording both choices and eye movements. While larger VLMs show some alignment with human choices, their attention patterns correlate poorly with human gaze, performing no better than a simple center‑bias baseline. Fine‑tuning VLMs on human choices improves choice alignment but not attention alignment, and training on human gaze improves attention correlation without affecting choice accuracy.

By Sumin Hong, Katsumi Ibaraki, Renee Shi, David Chiang, Toby Jia-Jun Li
arXiv AI
Jul 10

LEXIC: Lightweight Eye-tracking eXtension via Injected Complexity

arXiv:2607. 08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance.

By Sumin Lee, Kyeonghun Kim, Subeen Lee, Jiwon Yang, Tien Nguyen, Ken Ying-Kai Liao, Nam-Joon Kim