arXiv AI June 16, 2026 By Beidi Zhao, Wenlong Deng, Xinting Liao, Yushu Li, Nazim Shaikh, Yao Nie, Xiaoxiao Li

When RAG Hurts: Diagnosing and Mitigating Attention Distraction in Retrieval-Augmented LVLMs

arXiv:2602. 00344v2 Announce Type: replace-cross Abstract: While Retrieval-Augmented Generation (RAG) is one of the dominant paradigms for enhancing Large Vision-Language Models (LVLMs) on knowledge-based VQA tasks, recent work attributes RAG failures to insufficient attention towards the retrieved context, proposing to reduce the attention allocated to image tokens.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 7

Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

arXiv:2604. 01280v2 Announce Type: replace-cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires Multimodal Large Language Models (MLLMs) to identify and combine fine-grained visual cues with retrieved textual evidence.

By Marco Morini, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

llms nlp multimodal benchmarks

arXiv AI

Jul 9

T2T-VICL: Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs

arXiv:2511. 16107v3 Announce Type: replace-cross Abstract: Visual in-context learning (VICL) solves visual tasks by conditioning on a few input-output demonstrations without any model training.

By Shao-Jun Xia, Huixin Zhang, Zhengzhong Tu

llms multimodal safety

arXiv Computer Vision

3d ago

Composition, Not Conversation: VLMs Lose the Scene, Not the Thread

arXiv:2609.38368v1 Announce Type: new Abstract: Vision-language models (VLMs) increasingly reason over visual evidence that is cropped, segmented, retrieved, or revealed over time. Yet most VQA bench...

By L. D. M. S. Sai Teja, Ufaq Khan, N. Siva Gopala Krishna, Satyajit Tourani, Ashshak Sharifdeen, Fida Mohammad Thoker, Bernard Ghanem, Muhammad Haris Khan

llms multimodal benchmarks

arXiv Computer Vision

Aug 28

Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Information

The paper introduces Visual Retrieval Heads (VRHs), a small fraction of attention heads in vision‑language models that are causally responsible for grounding text descriptions to image regions. By recasting head‑scoring methods and evaluating across eleven VLMs and five benchmarks, the authors show that masking the top 20 VRHs can drop grounding accuracy by up to 80 percentage points, while random masking has little effect. VRHs generalize across various visual reference tasks, preserve output format while corrupting localization, and transfer causally across models sharing an LLM backbone.

By Chanho Park, Daehyeon Choi, Jihyun Lee, Minhyuk Sung

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 9

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

arXiv:2607. 07179v1 Announce Type: cross Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents.

By Miguel Lopez-Duran, Elena Marrero, Julian Fierrez, Marta Robledo-Moreno, Ruben Vera-Rodriguez, Daniel DeAlcala, Aythami Morales, Ruben Tolosana, Oscar Delgado, Alvaro Ortigosa, Javier Ortega-Garcia

llms nlp fine-tuning multimodal

arXiv Computer Vision

Sep 22

Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Act

arXiv:2609.22588v1 Announce Type: new Abstract: Vision-language models (VLMs) perform strongly on visual question answering benchmarks, yet often make decisions that contradict visual evidence they h...

By Yuyang Dai, Bofei Huang, Hongbo Zhang, Haoran Xie

llms nlp multimodal benchmarks

One email a morning, machine-written

Related stories

Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

T2T-VICL: Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs

Composition, Not Conversation: VLMs Lose the Scene, Not the Thread

Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Information

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Act