The paper introduces Visual Retrieval Heads (VRHs), a small fraction of attention heads in vision‑language models that are causally responsible for grounding text descriptions to image regions. By recasting head‑scoring methods and evaluating across eleven VLMs and five benchmarks, the authors show that masking the top 20 VRHs can drop grounding accuracy by up to 80 percentage points, while random masking has little effect. VRHs generalize across various visual reference tasks, preserve output format while corrupting localization, and transfer causally across models sharing an LLM backbone.
By Chanho Park, Daehyeon Choi, Jihyun Lee, Minhyuk Sung
The paper presents a matched‑budget audit framework for evaluating recaptioned image‑text supervision distributions, which are created by applying a captioning policy, captioner, and source corpus to generate dense descriptions for text‑to‑image training. The framework operates under a fixed text budget of 64 tokens and produces a five‑axis profile covering prompt coverage, image‑conditioned faithfulness, and caption surface quality, using controllable basic units as a common claim metric. The authors apply the framework to seven paired comparisons across five public corpora, demonstrating improvements in supported claim units and revealing consistent long‑vs‑dense trade‑offs across different judges and budgets, and they release the audited multi‑source recap corpus and audit artifacts.
By Giyeong Oh, Junghun Park, Yuhan Bae, Youngjae Yu
arXiv:2608.21762v1 Announce Type: cross
Abstract: Vision-language models (VLMs) fail many detail-centric questions for a concrete reason: the answer is visible in the image, yet lost after the image...
By Jinchang Zhu, Rong Fu, Yi Ding, Chenghao Wu, Ying Liu, Menglin Yang
The paper demonstrates that vision‑language models (VLMs) possess a small set of attention heads, called Visual Retrieval Heads (VRHs), that are causally responsible for linking text prompts to specific image regions. By adapting head‑scoring techniques from language models, the authors identify VRHs as the heads whose attention from output prediction tokens, summed over the ground‑truth referent region, most reliably indicates causal grounding. Experiments across eleven VLMs and five referring‑expression benchmarks show that masking the top 20 VRHs can drop grounding accuracy by up to 80 percentage points, while random masking has little effect, and that VRHs generalize across diverse visual tasks and transfer across models sharing an LLM backbone.
The paper introduces EviSpec, a training‑free compiler that generates complementary evidence specifications to improve high‑resolution multimodal large language models (MLLMs). By explicitly guiding visual search with structured evidence specifications, EviSpec achieves significant relative gains—up to 14.8% over random evidence—across five MLLMs and three benchmarks, and also sets new state‑of‑the‑art results on VQA and hallucination‑focused tasks.
By Zhongkuan Mao, Wenzhuo Zhao, Xianjie Liu, Yidong Wang, Zhao Gao, Ronghao Xian, Yao Jiang, Yi Zhang, Liangjian Wen, Keren Fu
arXiv:2609.00591v1 Announce Type: new
Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level c...
By Suryaansh Jain, Rahasya Barkur, Vishal G, Ryan Rossi, Franck Dernoncourt, Jack Wang, Koustava Goswami, Nedim Lipka, Puneet Mathur, Samyadeep Basu, Seunghyun Yoon
ET‑Prune is a training‑free framework that dynamically allocates visual token budgets in multimodal large language models based on question‑conditioned evidence. It protects text‑like spatial regions, converts evidence uncertainty into a token floor, and progressively prunes concentrated evidence while retaining diffuse or text‑dense tokens. In six backbone‑benchmark comparisons, ET‑Prune matches or outperforms other pruned methods while keeping roughly half the visual tokens, achieving notable gains on OCRBench‑v2 and MMBench v1.1.
By Zizhong Ding, Junxian Li, Kai Liu, Shaoqiu Zhang, Xiao Xiao, Linghe Kong, Yulun Zhang
arXiv:2511. 16107v3 Announce Type: replace-cross Abstract: Visual in-context learning (VICL) solves visual tasks by conditioning on a few input-output demonstrations without any model training.
By Shao-Jun Xia, Huixin Zhang, Zhengzhong Tu
arXiv:2605.26380v2 Announce Type: replace-cross
Abstract: Frontier multimodal large language models (MLLMs) have been reported to achieve over 90\% accuracy on fine-grained perception benchmarks. How...
By Jingru Chen, Yiming Liu, Mingtao Chen, Sijie Chen, Richeng Xuan, Liang Yang, Zhichao Hu, Fanyang Lu
The paper introduces AVA, a training‑free framework that improves interleaved multimodal in‑context learning for frozen large vision‑language models. AVA decomposes the interleaved context into per‑sample threads, selects influential demonstrations while removing redundancy, and calibrates consistency using a weak sibling model’s confidence. Across five vision‑language benchmarks, AVA raises performance by up to 2.4 points, narrows the visual performance gap on ScienceQA‑IMG, and remains robust even with 30% noisy demonstrations, all while keeping the LVLM frozen and adding only modest latency.
By Yigit Yilmaz, Elena Petrova, Mehmet Kaya, Lucia Rossi, Amir Rahman
arXiv:2609.36893v1 Announce Type: new
Abstract: Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy,...
By Zhenwen Ji, Lei Jin, Shanyong Wang, Jiaming Lu, Chengqiang Lu, Yi Wu, Yao Hu, Lizhen Cui, Yanyu Xu
arXiv:2605.27243v3 Announce Type: replace
Abstract: Large vision-language models increasingly rely on long-context modeling to reason over documents, hour-level videos, and long-horizon agent traject...
By Aaron Branson Cigres Li, Zhaowei Wang, Yu Zhao, Yiming Du, Haobo Li, Xiyu Ren, Ginny Wong, Simon See, Lishu Luo, Haodong Duan, Pasquale Minervini, Yangqiu Song