arXiv AI

Loud and Clear: Making Visual Evidence Count in Multimodal Document RAG

The paper introduces Loud and Clear (LoC), a method that enhances multimodal document retrieval-augmented generation by fully utilizing visual token capacity and refining retrieved pages to highlight evidence. LoC combines higher-resolution pages with OCR of query-relevant regions, achieving significant accuracy gains over eight prior methods on multiple benchmarks. The study demonstrates that the way retrieved pages are presented to the generator is a critical bottleneck beyond retrieval itself.

arXiv AI
Sep 2

MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval

MIDR (Multimodal Indexing for Document Retrieval) is a training‑free framework that enriches document indexes by converting rendered pages into verified textual fields with a multimodal LLM, then indexing those fields with BM25F and optionally fusing with dense retrieval. By shifting multimodal reasoning to index time, MIDR enables text‑centric serving while retaining multimodal evidence, achieving a 23.0% relative gain over BM25 on ViDoRe V3 and outperforming ColQwen2.5 on several domains with significantly smaller index memory and lower query latency.

By Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro, Yong Zhuang, Ozan Irsoy
arXiv Computation and Language
Aug 28

ET-Prune: Evidence-Aware Dynamic Budgeting for Visual Token Pruning in Text-Rich MLLMs

ET‑Prune is a training‑free framework that dynamically allocates visual token budgets in multimodal large language models based on question‑conditioned evidence. It protects text‑like spatial regions, converts evidence uncertainty into a token floor, and progressively prunes concentrated evidence while retaining diffuse or text‑dense tokens. In six backbone‑benchmark comparisons, ET‑Prune matches or outperforms other pruned methods while keeping roughly half the visual tokens, achieving notable gains on OCRBench‑v2 and MMBench v1.1.

By Zizhong Ding, Junxian Li, Kai Liu, Shaoqiu Zhang, Xiao Xiao, Linghe Kong, Yulun Zhang
arXiv Computer Vision
Sep 22

Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion

arXiv:2609.24220v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presen...

By Uday Allu (AI Research Team Yellow.ai), Abhivanth Sivaprakash (AI Research Team Yellow.ai), Pratik Singh (AI Research Team Yellow.ai), Aman Manocha (AI Research Team Yellow.ai)
arXiv Computer Vision
Sep 1

Can Retrieval Heads See Images? Multimodal Retrieval Heads in Long-Context Vision-Language Models

arXiv:2605.27243v3 Announce Type: replace Abstract: Large vision-language models increasingly rely on long-context modeling to reason over documents, hour-level videos, and long-horizon agent traject...

By Aaron Branson Cigres Li, Zhaowei Wang, Yu Zhao, Yiming Du, Haobo Li, Xiyu Ren, Ginny Wong, Simon See, Lishu Luo, Haodong Duan, Pasquale Minervini, Yangqiu Song
arXiv Computer Vision
Oct 5

Found but Not Read: When Extracted Text Closes the Retrieval-Reading Gap in Document Vision-Language Models

The paper investigates the gap between retrieval and reading in document vision‑language models, showing that even when the correct page is retrieved, the model often fails to use the text on that page. By comparing answers derived from page images alone versus images plus extracted OCR text, the authors demonstrate that adding OCR text can improve strict accuracy by 13 to 16 points on their FoveDoc‑Bench benchmark. The study also reveals that OCR benefits textual evidence but not charts or figures, and that its advantage diminishes as retrieval quality worsens.

By Qingtao Xia, Siyao Cheng, Jiahua Bao, Jiaxing Du, Jie Liu
arXiv AI
Jun 30

PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation

arXiv:2606. 28344v1 Announce Type: cross Abstract: Augmenting large language models (LLMs) with retrieved web text has become a dominant paradigm, yet the web is not natively textual: existing systems depend on complex parsing pipelines that linearize HTML and discard layout, visual structure, and formatting.

By Yichuan Wang, Zhifei Li, Zirui Wang, Paul Teiletche, Lesheng Jin, Matei Zaharia, Joseph E. Gonzalez, Sewon Min