arXiv Computation and Language
Learning to Refer from Estimated Listener Gaze
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Sep 17
Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX
arXiv:2609.18011v1 Announce Type: new Abstract: In collaborative tasks with asymmetric information, participants coordinate their understanding through interaction. We ask whether gaze provides evide...
arXiv Computer Vision
Oct 5
Gaze Attention: Query-Adaptive Visual Routing for Efficient Multimodal LLMs
Gaze Attention is a new mechanism for multimodal large language models that selectively focuses on relevant visual regions during each generation step, rather than attending to all visual tokens. By grouping tokens into spatial regions and using learnable context tokens to retain global information, it reduces attention computation and visual key‑value entries by up to 90%. Experiments on 13 image and 6 video benchmarks show that Gaze Attention matches or outperforms dense‑attention baselines while using fewer visual resources.
Hugging Face Trending Papers
Jul 9
VEGAS: Human-Aligned Video Caption Evaluation via Gaze
Vision-language models excel at video captioning, yet typically generate descriptions that fail to capture individual viewers' attention. We propose VEGAS (Video caption Evaluation via GAze Score), a training-free metric that leverages test-time gaze to sample personalized, attention-aligned text.
arXiv AI
Jul 10
VEGAS: Human-Aligned Video Caption Evaluation via Gaze
arXiv:2607. 08489v1 Announce Type: cross Abstract: Vision-language models excel at video captioning, yet typically generate descriptions that fail to capture individual viewers' attention.
arXiv Computation and Language
Sep 1
Which one is banana man? Evaluating vision-language models in multi-turn pragmatic interpretation
The study examines how vision‑language models handle multi‑turn pragmatic interpretation in iterated reference games, where participants repeatedly identify novel referents using language. Researchers compared human performance with that of several models, manipulating context by varying its amount, order, and relevance. While humans consistently performed well, the models could use prior context but struggled to build relevant context for effective interpretation, indicating missing core skills for efficient linguistic collaboration.
arXiv AI
Jul 10
LEXIC: Lightweight Eye-tracking eXtension via Injected Complexity
arXiv:2607. 08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance.