arXiv:2606. 07654v1 Announce Type: cross Abstract: Multi-vector visual document retrievers achieve strong fine-grained matching by representing each page with multiple vectors from deep Vision-Language Models (VLMs), but this design makes deployment expensive in both storage and computational overhead.
arXiv:2608.29951v1 Announce Type: new Abstract: Multi-modal late-interaction retrievers achieve strong retrieval on visually rich documents by representing each page as per patch embeddings and match...
arXiv:2609.21018v1 Announce Type: new Abstract: Recent visual document retrieval (VDR) systems such as ColPali use multi-vector page embeddings, in which patch-level vectors enable fine-grained evide...
arXiv:2606. 04240v1 Announce Type: cross Abstract: Retrieval over visually-rich documents, pages that interleave text with figures, tables, and charts, is essential for multimodal retrieval-augmented generation, yet most retrievers still discard the visual channel.
arXiv:2609.37225v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have shown strong potential for universal multimodal representation learning. However, existing methods eith...
arXiv:2608.30163v1 Announce Type: cross Abstract: Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, ex...
arXiv:2609.07262v1 Announce Type: cross Abstract: Late-interaction visual document retrievers preserve fine-grained page evidence by storing many token embeddings per page, but the resulting storage...
CausalEmbed is an auto‑regressive method for generating compact multi‑vector embeddings in visual document retrieval. By using iterative margin loss during contrastive training, it reduces the number of visual tokens needed by 30‑155× while keeping performance competitive across different backbones and benchmarks. The approach offers efficient training, scalable test‑time performance, and a flexible scaling strategy for multi‑vector representations.
The paper introduces ReT-2, a unified retrieval model that handles multimodal queries containing both images and text and searches across multimodal document collections. It employs a recurrent Transformer architecture with LSTM-inspired gating to integrate information across layers and modalities, capturing fine-grained visual and textual details. Evaluations on M2KR and M-BEIR benchmarks show state‑of‑the‑art performance, faster inference, and lower memory usage, and the model also boosts downstream tasks in retrieval‑augmented generation pipelines.
The paper introduces Invoice Haystack, a benchmark of 1,500 anonymized invoices and 200 question‑answer pairs that tests document retrieval and visual question answering under strong visual homogeneity. It shows that existing benchmarks suffer from embedding collapse, with Invoice Haystack’s mean pairwise cosine similarity at 0.73 versus 0.38 and 0.31 in DocHaystack and InfoHaystack. The authors propose VL‑RAG, a hybrid retrieval‑augmented generation framework that combines text and visual embeddings and a VLM‑based verification filter, achieving 60.0% Recall@1 on Invoice Haystack‑500 and improving performance on other benchmarks.
The paper introduces AllocEmbed, an allocate‑then‑embed framework that reallocates a fixed visual‑input budget across more video frames to improve retrieval performance. A lightweight allocator uses low‑cost previews to assign frame‑wise resolutions before the embedding backbone, preserving detail where it most benefits retrieval while reducing visual cost elsewhere. Retrieval‑Driven Policy Optimization (RDPO) learns the allocator directly from retrieval feedback, and the method integrates with existing systems without modifying the embedding model.
arXiv:2608.05707v2 Announce Type: replace Abstract: Frame selection is essential for applying Large Multimodal Models (LMMs) to long videos due to severe frame redundancy and limited context windows....
ViSAR is a training‑free, adaptive‑k retrieval method for Visual Document Question Answering that operates directly in the embedding space to build a query‑conditioned page‑level similarity matrix. By dynamically selecting the number of pages to retrieve based on query relevance, ViSAR reduces Retrieval‑Augmented Generation latency by up to 58.7% while maintaining or improving answer accuracy across multiple encoders and Large Vision‑Language Models. The structure of the similarity matrix also correlates with answer accuracy, indicating potential for retrieval quality‑aware document understanding.