arXiv:2610.00922v1 Announce Type: new
Abstract: Gaze estimation under natural head-eye motion underpins applications from driver monitoring to human-computer interaction. Single-frame methods predict...
By Jungmin Lee, Niamat Ullah, Yoseob Han
StreamGaze is a new benchmark that evaluates how well Multimodal Large Language Models (MLLMs) use human gaze signals for temporal and proactive reasoning in streaming videos. It introduces gaze‑guided tasks covering past, present, and future responses, and employs a QA generation pipeline that aligns egocentric videos with raw gaze trajectories. The benchmark reveals significant gaps between current MLLMs and human performance, highlighting limitations in gaze‑based temporal reasoning, intent modeling, and proactive prediction.
By Daeun Lee, Subhojyoti Mukherjee, Branislav Kveton, Ryan A. Rossi, Viet Dac Lai, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Mohit Bansal
arXiv:2610.28502v1 Announce Type: new
Abstract: Smart glasses can help users understand their activities and surroundings, but small vision-language models must preserve useful evidence within limite...
By Trong-Thang Pham, Anh Nguyen, Hien Van Nguyen, Ngan Le
EgoM2P is a masked‑modeling framework that learns from temporally‑aware multimodal tokens to train a large, general‑purpose model for egocentric 4D understanding. It supports multitasking across diverse egocentric perception and synthesis tasks—such as gaze prediction, egocentric camera tracking, monocular depth estimation, and conditional egocentric video synthesis—while matching or surpassing specialist models at a fraction of the computational cost. The authors plan to fully open‑source EgoM2P to advance egocentric vision research.
By Gen Li, Yutong Chen, Yiqian Wu, Kaifeng Zhao, Marc Pollefeys, Siyu Tang
arXiv:2608.22926v1 Announce Type: new
Abstract: Gaze is increasingly used as an input signal for vision and multimodal models, yet no consensus exists on how to represent it across datasets. Raw trac...
By Virmarie Maquiling, Zhuojiang Cai, Enkelejda Kasneci
Gaze Attention is a new mechanism for multimodal large language models that selectively focuses on relevant visual regions during each generation step, rather than attending to all visual tokens. By grouping tokens into spatial regions and using learnable context tokens to retain global information, it reduces attention computation and visual key‑value entries by up to 90%. Experiments on 13 image and 6 video benchmarks show that Gaze Attention matches or outperforms dense‑attention baselines while using fewer visual resources.
By Junha Song, Byeongho Heo, Geonmo Gu, Jaegul Choo, Dongyoon Han, Sangdoo Yun