arXiv Computer Vision

MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval

arXiv Computation and Language
6d ago

OpticalRAG: Pixel-Space Compression for Token-Efficient Retrieval-Augmented Generation

OpticalRAG is a retrieval‑augmented generation framework that compresses documents by rendering them as images and encoding them with a frozen vision encoder. It retrieves visual tokens using an Encoder‑Bridged Retrieval mechanism and condenses them into as few as 16 query‑conditioned prefix tokens via Query‑Driven Distillation, avoiding costly cross‑modal training. The method achieves state‑of‑the‑art compression‑aware accuracy on LongBench‑v2, BAMBOO, and LooGLE‑v2 while adding only 16 tokens to the LLM input.

By Senhao Liu, Yuheng Zhang, Chunyu Wei, Yueguo Chen, Xinran Zhang
arXiv AI
Jun 9

MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework

arXiv:2606. 07654v1 Announce Type: cross Abstract: Multi-vector visual document retrievers achieve strong fine-grained matching by representing each page with multiple vectors from deep Vision-Language Models (VLMs), but this design makes deployment expensive in both storage and computational overhead.

By Haowen Xiang, Yibo Yan, Jiahao Huo, Yu Huang, Yi Cao, Mingdong Ou, Xuming Hu
Hugging Face Trending Papers
Aug 9

AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrieval

Multi-vector vision-language retrievers enable fine-grained Visual Document Retrieval (VDR) through late interaction, but storing and scoring hundreds of visual patch embeddings per page incurs substantial overhead. Existing training-free methods rely on pruning or merging: pruning degrades sharply under aggressive compression, whereas merging does not explicitly prioritize important regions when forming representatives.

arXiv AI
Oct 16

Loud and Clear: Making Visual Evidence Count in Multimodal Document RAG

The paper introduces Loud and Clear (LoC), a method that enhances multimodal document retrieval-augmented generation by fully utilizing visual token capacity and refining retrieved pages to highlight evidence. LoC combines higher-resolution pages with OCR of query-relevant regions, achieving significant accuracy gains over eight prior methods on multiple benchmarks. The study demonstrates that the way retrieved pages are presented to the generator is a critical bottleneck beyond retrieval itself.

By Eunchan Jo, Seounghyun Jang, Sangwoo Mo
arXiv Computer Vision
Sep 4

Invoice Haystack: Benchmarking Document Retrieval and Visual Question Answering Under Strong Visual Homogeneity

The paper introduces Invoice Haystack, a benchmark of 1,500 anonymized invoices and 200 question‑answer pairs that tests document retrieval and visual question answering under strong visual homogeneity. It shows that existing benchmarks suffer from embedding collapse, with Invoice Haystack’s mean pairwise cosine similarity at 0.73 versus 0.38 and 0.31 in DocHaystack and InfoHaystack. The authors propose VL‑RAG, a hybrid retrieval‑augmented generation framework that combines text and visual embeddings and a VLM‑based verification filter, achieving 60.0% Recall@1 on Invoice Haystack‑500 and improving performance on other benchmarks.

By Heethanjan Kanagalingam, Thenukan Pathmanathan, Mokeeshan Vathanakumar, Basim Azam, Sarah Monazam Erfani, Naveed Akhtar
arXiv AI
Sep 2

MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval

MIDR (Multimodal Indexing for Document Retrieval) is a training‑free framework that enriches document indexes by converting rendered pages into verified textual fields with a multimodal LLM, then indexing those fields with BM25F and optionally fusing with dense retrieval. By shifting multimodal reasoning to index time, MIDR enables text‑centric serving while retaining multimodal evidence, achieving a 23.0% relative gain over BM25 on ViDoRe V3 and outperforming ColQwen2.5 on several domains with significantly smaller index memory and lower query latency.

By Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro, Yong Zhuang, Ozan Irsoy