arXiv:2606. 31719v1 Announce Type: cross Abstract: In collaborative dialogue, shared perception does not guarantee shared interpretation.
arXiv:2609.14207v1 Announce Type: new Abstract: We propose to finetune vision-language models to generate more pragmatically optimal referring expressions by transforming observations of incremental...
The study compares human and vision‑language model (VLM) responses to cross‑modal association tasks, using identical stimuli (a pseudo‑word and two images) and recording both choices and eye movements. While larger VLMs show some alignment with human choices, their attention patterns correlate poorly with human gaze, performing no better than a simple center‑bias baseline. Fine‑tuning VLMs on human choices improves choice alignment but not attention alignment, and training on human gaze improves attention correlation without affecting choice accuracy.
The study examines how reader proficiency influences the relationship between layer-wise surprisal from large language models (LLMs) and eye-tracking gaze measures. Using the MECO L2 corpus, researchers compared high- and low-proficiency readers on first-pass gaze duration (FPGD) and total gaze duration (TGD), finding that lower-proficiency readers exhibit deeper Predictive Depth for FPGD, while TGD shows deeper Predictive Depth across both groups. The results suggest that the distribution of predictive power across LLM layers relates to the timing and breadth of reading processes and varies with reader proficiency.
GazeFS is a model that predicts and stabilizes target‑centered gaze trajectories using a variable‑length gaze‑head history, without requiring target information during inference. It maps this history to the next target‑center direction and a short‑horizon Search/Focus estimate, improving focus target centering and reducing residual gaze error. Across 7,960 acquisition episodes from 30 participants, GazeFS reduces Focus episode bias, dispersion, and P90 target error by 0.182°, 0.257°, and 0.400°, respectively, while maintaining high phase‑balanced accuracy and AUPRC.
arXiv:2608. 16514v1 Announce Type: cross Abstract: Human visual search is serial: the fovea must land on a candidate to confirm it, and those landings form a scanpath.
The study investigates how gaze and speech cues, together with perceived interpersonal closeness, predict turn‑taking outcomes in free four‑person conversations. Using the GaMMA corpus, logistic regression models were trained on interpretable features such as gaze transition motifs, entropy, addressee identity, mutual gaze, and speaker loudness to classify floor‑transfer events as gaps or overlaps. Results show that gaze features alone capture predictive structure, and combining them with loudness yields a robust classifier (ROC AUC = 0.76 ± 0.04) that remains effective even under noisy conditions.
arXiv:2610.15129v1 Announce Type: new Abstract: Human social perception is incremental: we infer intent before an utterance is complete, revise our interpretation as new cues arrive, and attend to sp...
arXiv:2607. 08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance.
arXiv:2606. 08081v1 Announce Type: cross Abstract: Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific conventions grounded in shared interaction history.
The study investigates how users switch roles in a human‑AI chess collaboration, using multimodal behavioral signals such as gaze and task‑specific features. Participants mostly retained their roles, but when they switched they showed more exploratory gaze and poorer move quality. A classifier trained on these signals achieved a PR‑AUC of 0.56, indicating that behavioral cues can predict role switches.
The paper investigates whether multimodal large language model (LLM) agents form partner‑specific conceptual pacts during repeated reference games. By comparing real agent dyads with a novel pseudo‑dyad baseline that preserves task structure but removes shared partner history, the authors show that humans compress descriptions and align labels through entrainment, whereas agents produce verbose, fixed‑effort descriptions with near‑ceiling label overlap regardless of partner. Thus, LLMs succeed in coordination without forming compact, history‑dependent referring expressions typical of human dialogue.