The paper introduces a controlled evaluation framework for interactive visual grounding in large vision-language models (LVLMs), examining how varying amounts of initial target information and dialogue affect performance. Experiments across four visual contexts and interaction protocols show that current LVLMs lag behind human baselines, especially when no initial description is given and information must be gathered through questions. The study also finds that LVLMs are poorly calibrated, often overestimating confidence, and that interactive grounding remains a significant challenge requiring visual matching, information seeking, and synthesis.
By Zhengxiang Wang, Owen Rambow
arXiv:2609.18011v1 Announce Type: new
Abstract: In collaborative tasks with asymmetric information, participants coordinate their understanding through interaction. We ask whether gaze provides evide...
By Nan Li, Albert Gatt, Massimo Poesio
arXiv:2609.00293v1 Announce Type: new
Abstract: We investigate how vision-language models (VLMs) handle context-memory conflicts; that is, situations in which the model is given information in contex...
By Athulith Paraselli, Etha Tianze Hua, Ellie Pavlick
Conversational DNA is a visual language and interactive atlas designed to explore human and AI dialogue by mapping speaker strands, communicative bases, and directed pairings. It visualizes speaker switching, response distance, and contribution length through adjustable helix geometry, and covers 151,489 episodes across eight corpora totaling 1.57 million source records. The system improves precision@5 on Molweni motif queries from 58.8% to 77.2% and demonstrates how annotation coverage affects perceived collection differences.
By Baihan Lin
arXiv:2606. 08081v1 Announce Type: cross Abstract: Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific conventions grounded in shared interaction history.
By Po-Ya Angela Wang, Chinmaya Mishra, Asl{\i} \"Ozy\"urek, Paula Rubio-Fern\'andez, Esam Ghaleb
The paper introduces a new task called compositional layout understanding, focusing on interpreting complex, multi-layer document and UI layouts that involve hierarchical relationships among visually entangled elements. It presents CoDeLayout, a VQA dataset of about 20,000 real-world layouts annotated with compositional element pairs and design intent, and identifies two main challenges for current vision‑language models: semantic drift between textual metadata and visual content, and structural ambiguity in hierarchical inter‑element relationships. To address these, the authors propose MASON, a post‑training paradigm that combines multimodal alignment and structural perception, achieving a 91.66% accuracy on CoDeLayout and outperforming full‑data direct fine‑tuning with only 30% of the training data.