arXiv Computation and Language By Aviral Joshi, Hanoz Bhathena, Max Nelson, Saket Sharma

Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines

Read the original on arXiv Computation and Language →

Re:CAP is a reference‑free audit loop for retrieval‑augmented generation (RAG) pipelines that probes for missing documents instead of enumerating all relevant ones. It identifies covered topics, generates probing questions, retrieves candidate documents, and uses an LLM judge to keep only those that add new information. On several benchmarks, Re:CAP recovers a significant portion of gold documents that flat BM25 or hybrid retrieval misses, and human evaluation shows most of these documents add new information.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language
Nov 2

Better Retrievers, Worse Labels: How Incomplete Relevance Judgments Distort Abstention Evaluation in Conversational RAG

The paper investigates how conversational Retrieval-Augmented Generation (RAG) benchmarks that rely on relevance judgments to determine context sufficiency can be misleading. By re‑running official baselines on the RETECO track and evaluating with both large‑language‑model judges and a blind audit, the authors show that stronger retrievers produce a higher proportion of contexts lacking gold documents yet still containing the answer, leading to a sharp drop in shortcut precision from 0.68 to 0.10. The study demonstrates that many false misses arise from hard negatives, gold content from other turns, or neighboring chunks, and that using retrieval scores alone as a sufficiency detector can underestimate answerability by up to 0.19 in AUROC.

By Ivan Kochergin, Vasily Konovalov
arXiv Computation and Language
Oct 16

Beyond Retrieval Failure: Understanding Why LLMs Miss Evidence in Multi-Hop RAG

The paper investigates why large language models (LLMs) miss evidence in multi‑hop retrieval‑augmented generation (RAG). It introduces ExecutionRAG, a training‑free register that tracks open sub‑questions and their supporting evidence, allowing the authors to distinguish between true retrieval failures and search drift. Using this method on 1,000 MuSiQue questions, the study shows a significant reduction in missed evidence but also reveals that other error types, such as premature resolutions and abstentions, increase, shifting the bottleneck from search guidance to evidence verification.

By Jiaqi Wei, Wenxuan Huang, Chengdong Xu, Xiang Zhang, Wanli Ouyang
arXiv AI
Jul 1

RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora

arXiv:2604. 19047v2 Announce Type: replace-cross Abstract: Existing QA benchmarks typically assume distinct documents with minimal overlap, yet real-world retrieval-augmented generation (RAG) systems operate on corpora such as financial reports, legal codes, and patents, where information is highly redundant and documents exhibit strong inter-document similarity.

By Hanjun Cho, Jay-Yoon Lee