arXiv Computation and Language

Cascaded Retrieval with Cross-Modal Query Refinement for Efficient Visual Document Search

arXiv Computer Vision
Aug 27

VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval

VisDocAgentBench is a closed‑corpus benchmark that evaluates static versus agentic retrieval for visually rich documents, using 2,375 pages from 100 documents and 120 queries that span direct, one‑bridge, and two‑bridge evidence structures. The benchmark includes semantic, relational, and visual queries, full‑document review, and hard‑negative validation. Results show that a strong visual retriever performs well on direct items but poorly on two‑bridge items, while agents improve performance, especially when using visual retrieval and iterative search capabilities.

By Lexiang Hu, Yanzhao Zhang, Mingxin Li, Dingkun Long, Yikang Li, Fuwei Zhang, Yisen Wang, Zhouchen Lin
arXiv Computer Vision
Sep 4

Invoice Haystack: Benchmarking Document Retrieval and Visual Question Answering Under Strong Visual Homogeneity

The paper introduces Invoice Haystack, a benchmark of 1,500 anonymized invoices and 200 question‑answer pairs that tests document retrieval and visual question answering under strong visual homogeneity. It shows that existing benchmarks suffer from embedding collapse, with Invoice Haystack’s mean pairwise cosine similarity at 0.73 versus 0.38 and 0.31 in DocHaystack and InfoHaystack. The authors propose VL‑RAG, a hybrid retrieval‑augmented generation framework that combines text and visual embeddings and a VLM‑based verification filter, achieving 60.0% Recall@1 on Invoice Haystack‑500 and improving performance on other benchmarks.

By Heethanjan Kanagalingam, Thenukan Pathmanathan, Mokeeshan Vathanakumar, Basim Azam, Sarah Monazam Erfani, Naveed Akhtar
arXiv Computation and Language
Aug 25

L\"etzCross: A Cross-Lingual Page-Level Benchmark for Multimodal Retrieval over Luxembourgish Documents

arXiv:2608.21714v1 Announce Type: new Abstract: Recent page-image retrievers such as ColPali have improved retrieval over visually rich documents, yet little is known about how they behave in cross-l...

By Omar El Bachyr, Fred Philippy, Laura Maria Bernardy, Saad Ezzini, Jacques Klein, Tegawende Bissyande
Hugging Face Trending Papers
Aug 9

AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrieval

Multi-vector vision-language retrievers enable fine-grained Visual Document Retrieval (VDR) through late interaction, but storing and scoring hundreds of visual patch embeddings per page incurs substantial overhead. Existing training-free methods rely on pruning or merging: pruning degrades sharply under aggressive compression, whereas merging does not explicitly prioritize important regions when forming representatives.

arXiv Computation and Language
Aug 27

Recurrence Meets Transformers for Universal Multimodal Retrieval

The paper introduces ReT-2, a unified retrieval model that handles multimodal queries containing both images and text and searches across multimodal document collections. It employs a recurrent Transformer architecture with LSTM-inspired gating to integrate information across layers and modalities, capturing fine-grained visual and textual details. Evaluations on M2KR and M-BEIR benchmarks show state‑of‑the‑art performance, faster inference, and lower memory usage, and the model also boosts downstream tasks in retrieval‑augmented generation pipelines.

By Davide Caffagni, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara
arXiv Computer Vision
1d ago

KoViDoRe: A Benchmark for Korean Visual Document Retrieval

KoViDoRe is a new benchmark for Korean visual document retrieval that addresses gaps in existing English‑centric datasets by including Korean documents with complex layouts such as tables, figures, and multi‑column structures. The dataset is built through a multi‑stage pipeline involving structured document parsing, synthetic query generation (summary‑based and context‑based), and human‑verified relevance mapping. Evaluation on KoViDoRe shows that current multimodal retrieval models struggle with Korean documents, especially when handling structured content and diverse query types, prompting the creation of a large‑scale training set, Ko‑VDR Train Public, to support model development.

By Yongbin Choi, Yongwoo Song, Mujeen Sung