arXiv Computation and Language
Learning to Refer from Estimated Listener Gaze
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
1d ago
Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX
arXiv:2609.18011v1 Announce Type: new Abstract: In collaborative tasks with asymmetric information, participants coordinate their understanding through interaction. We ask whether gaze provides evide...
Hugging Face Trending Papers
Jul 9
VEGAS: Human-Aligned Video Caption Evaluation via Gaze
Vision-language models excel at video captioning, yet typically generate descriptions that fail to capture individual viewers' attention. We propose VEGAS (Video caption Evaluation via GAze Score), a training-free metric that leverages test-time gaze to sample personalized, attention-aligned text.
arXiv AI
Jul 10
VEGAS: Human-Aligned Video Caption Evaluation via Gaze
arXiv:2607. 08489v1 Announce Type: cross Abstract: Vision-language models excel at video captioning, yet typically generate descriptions that fail to capture individual viewers' attention.
arXiv Computation and Language
Sep 1
Which one is banana man? Evaluating vision-language models in multi-turn pragmatic interpretation
The study examines how vision‑language models handle multi‑turn pragmatic interpretation in iterated reference games, where participants repeatedly identify novel referents using language. Researchers compared human performance with that of several models, manipulating context by varying its amount, order, and relevance. While humans consistently performed well, the models could use prior context but struggled to build relevant context for effective interpretation, indicating missing core skills for efficient linguistic collaboration.
arXiv AI
Jul 10
LEXIC: Lightweight Eye-tracking eXtension via Injected Complexity
arXiv:2607. 08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance.
arXiv AI
Sep 10
Emergent Goal-Directed Attention in Large Vision-Language Models
arXiv:2609.05517v1 Announce Type: cross Abstract: Human observers prioritize visual information according to task goals. Most computational models of naturalistic viewing are gaze-trained for free vi...