arXiv AI

Query-Aware Token Budgeting for Efficient Late-Interaction Visual Document Retrieval

arXiv Computation and Language
Oct 21

FLINT: Fixed-budget Late Interaction

arXiv:2610.27273v1 Announce Type: cross Abstract: Multi-vector retrieval models such as ColBERT often outperform single-vector models of comparable scale by matching queries and documents at the toke...

By Himank Handa, Negar Arabzadeh, Matei Zaharia
arXiv AI
Sep 2

MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval

MIDR (Multimodal Indexing for Document Retrieval) is a training‑free framework that enriches document indexes by converting rendered pages into verified textual fields with a multimodal LLM, then indexing those fields with BM25F and optionally fusing with dense retrieval. By shifting multimodal reasoning to index time, MIDR enables text‑centric serving while retaining multimodal evidence, achieving a 23.0% relative gain over BM25 on ViDoRe V3 and outperforming ColQwen2.5 on several domains with significantly smaller index memory and lower query latency.

By Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro, Yong Zhuang, Ozan Irsoy
Hugging Face Trending Papers
Aug 9

AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrieval

Multi-vector vision-language retrievers enable fine-grained Visual Document Retrieval (VDR) through late interaction, but storing and scoring hundreds of visual patch embeddings per page incurs substantial overhead. Existing training-free methods rely on pruning or merging: pruning degrades sharply under aggressive compression, whereas merging does not explicitly prioritize important regions when forming representatives.

arXiv Computer Vision
Aug 27

VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval

VisDocAgentBench is a closed‑corpus benchmark that evaluates static versus agentic retrieval for visually rich documents, using 2,375 pages from 100 documents and 120 queries that span direct, one‑bridge, and two‑bridge evidence structures. The benchmark includes semantic, relational, and visual queries, full‑document review, and hard‑negative validation. Results show that a strong visual retriever performs well on direct items but poorly on two‑bridge items, while agents improve performance, especially when using visual retrieval and iterative search capabilities.

By Lexiang Hu, Yanzhao Zhang, Mingxin Li, Dingkun Long, Yikang Li, Fuwei Zhang, Yisen Wang, Zhouchen Lin
arXiv Computer Vision
Nov 2

Defining Page-Centric Visual Document Retrieval and Evaluating Existing Benchmarks

The paper introduces page‑centric Visual Document Retrieval (VDR), treating pages as the primary indexing units while leveraging document context for relevance assessment. It distinguishes page and document retrieval granularities and three retrieval paradigms—independent‑item, evidence‑set, and iterative—then evaluates 42 existing benchmarks across eight criteria. The analysis reveals that most current VDR research focuses on page‑level independent‑item retrieval, with limited support for document‑level, evidence‑set, and iterative settings, and that realistic user queries and graded relevance judgments are rarely used, highlighting gaps between application needs and evaluation protocols.

By Chongzhi Zhang, Yuqi Zuo, Aixin Sun