arXiv AI By Chaeho Lee, Junyup Kim, Wansoo Kim, Young Min Jung, Sang Ho Lee

Leveraging Psychophysical Attentional Distribution for Gaze-Augmented Reward Modeling

Read the original on arXiv AI →

The paper proposes a new reward modeling approach that uses a spatially graded distribution of visual attention centered on the current gaze location, rather than just fixation points. By incorporating this gaze distribution into reinforcement learning from human feedback, the authors achieve higher preference prediction accuracy compared to a baseline. The results also qualitatively align with known patterns of human attention during reading, indicating that the model captures cognitively meaningful aspects of attention allocation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 30

Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models

The study compares human and vision‑language model (VLM) responses to cross‑modal association tasks, using identical stimuli (a pseudo‑word and two images) and recording both choices and eye movements. While larger VLMs show some alignment with human choices, their attention patterns correlate poorly with human gaze, performing no better than a simple center‑bias baseline. Fine‑tuning VLMs on human choices improves choice alignment but not attention alignment, and training on human gaze improves attention correlation without affecting choice accuracy.

By Sumin Hong, Katsumi Ibaraki, Renee Shi, David Chiang, Toby Jia-Jun Li