Visual document retrieval is a critical component of multimodal retrieval-augmented generation, aiming to identify query-relevant pages from document collections where evidence is distributed across t...
Visual document retrieval has recently become increasingly important in applications such as enterprise search, scientific literature discovery, and retrieval-augmented generation. These applications depend on efficiently identifying query-relevant pages across large collections of visually rich documents.
VisDocAgentBench is a closed‑corpus benchmark that evaluates static versus agentic retrieval for visually rich documents, using 2,375 pages from 100 documents and 120 queries that span direct, one‑bridge, and two‑bridge evidence structures. The benchmark includes semantic, relational, and visual queries, full‑document review, and hard‑negative validation. Results show that a strong visual retriever performs well on direct items but poorly on two‑bridge items, while agents improve performance, especially when using visual retrieval and iterative search capabilities.
ViSAR is a training‑free, adaptive‑k retrieval method for Visual Document Question Answering that operates directly in the embedding space to build a query‑conditioned page‑level similarity matrix. By dynamically selecting the number of pages to retrieve based on query relevance, ViSAR reduces Retrieval‑Augmented Generation latency by up to 58.7% while maintaining or improving answer accuracy across multiple encoders and Large Vision‑Language Models. The structure of the similarity matrix also correlates with answer accuracy, indicating potential for retrieval quality‑aware document understanding.
arXiv:2506. 08774v2 Announce Type: replace-cross Abstract: Different machine learning models can represent the same underlying concept in different ways.
The paper introduces CERES, a closed‑loop multimodal indexing framework that addresses semantic collapse in multimodal generation by building a three‑level semantic pyramid and using scale‑routed cross‑attention to generate images that remain retrievable by their original queries. CERES employs a co‑occurrence‑aware router, a lightweight U‑Net generator, and a soft‑Jaccard coverage objective to ensure generated images cover the intended concepts, verified by re‑indexing with a frozen vision‑language model and an external DINOv2 probe. Experiments on four pansharpening benchmarks show state‑of‑the‑art performance, especially under extreme scale variation, and significant improvements in concept‑query retrieval and image‑text ranking metrics.
The paper introduces Invoice Haystack, a benchmark of 1,500 anonymized invoices and 200 question‑answer pairs that tests document retrieval and visual question answering under strong visual homogeneity. It shows that existing benchmarks suffer from embedding collapse, with Invoice Haystack’s mean pairwise cosine similarity at 0.73 versus 0.38 and 0.31 in DocHaystack and InfoHaystack. The authors propose VL‑RAG, a hybrid retrieval‑augmented generation framework that combines text and visual embeddings and a VLM‑based verification filter, achieving 60.0% Recall@1 on Invoice Haystack‑500 and improving performance on other benchmarks.
The paper introduces Concept-Driven Domain Adaptation (CDDA), a three-stage framework that adapts vision‑language models for concept‑to‑example video retrieval in educational settings. CDDA first structures textual embeddings using textbook and teacher‑handbook concept pairs, then transfers this geometry to documentary visuals with a frozen visual encoder, and finally jointly fine‑tunes both encoders with sparse visual concept supervision. On a middle‑school physics benchmark, CDDA outperforms several multimodal baselines in retrieving concept‑driven moments while preserving concrete image‑text alignment.
arXiv:2608.21450v1 Announce Type: new Abstract: Knowledge-Based Visual Question Answering (KB-VQA) relies on retrieving external information to answer queries involving long-tail entities. However, e...
arXiv:2606. 10572v1 Announce Type: new Abstract: External memory effectively grounds large language models (LLMs) and vision-language models (VLMs)-based question answering (QA) in relevant multimodal evidence.
arXiv:2607. 20556v1 Announce Type: new Abstract: In large-scale text analysis tasks, pre-trained language models are often used to embed text corpora for downstream analysis.
arXiv:2602.09839v2 Announce Type: replace Abstract: Existing multimodal retrieval benchmarks largely emphasize semantic matching on daily-life images and offer limited diagnostics of professional kno...