The paper investigates how conversational Retrieval-Augmented Generation (RAG) benchmarks that rely on relevance judgments to determine context sufficiency can be misleading. By re‑running official baselines on the RETECO track and evaluating with both large‑language‑model judges and a blind audit, the authors show that stronger retrievers produce a higher proportion of contexts lacking gold documents yet still containing the answer, leading to a sharp drop in shortcut precision from 0.68 to 0.10. The study demonstrates that many false misses arise from hard negatives, gold content from other turns, or neighboring chunks, and that using retrieval scores alone as a sufficiency detector can underestimate answerability by up to 0.19 in AUROC.
By Ivan Kochergin, Vasily Konovalov
The paper investigates why large language models (LLMs) miss evidence in multi‑hop retrieval‑augmented generation (RAG). It introduces ExecutionRAG, a training‑free register that tracks open sub‑questions and their supporting evidence, allowing the authors to distinguish between true retrieval failures and search drift. Using this method on 1,000 MuSiQue questions, the study shows a significant reduction in missed evidence but also reveals that other error types, such as premature resolutions and abstentions, increase, shifting the bottleneck from search guidance to evidence verification.
By Jiaqi Wei, Wenxuan Huang, Chengdong Xu, Xiang Zhang, Wanli Ouyang
arXiv:2609.38021v1 Announce Type: cross
Abstract: We evaluate an auditable long-term memory system on LongMemEval-S. Its retrieval chain uses hybrid candidate retrieval, cross-encoder reranking, cove...
By Christopher J. Chanhnourack
arXiv:2604. 19047v2 Announce Type: replace-cross Abstract: Existing QA benchmarks typically assume distinct documents with minimal overlap, yet real-world retrieval-augmented generation (RAG) systems operate on corpora such as financial reports, legal codes, and patents, where information is highly redundant and documents exhibit strong inter-document similarity.
By Hanjun Cho, Jay-Yoon Lee
arXiv:2606.11350v2 Announce Type: replace
Abstract: Retrieval-augmented generation (RAG) degrades when applied to large, heterogeneous document collections. In RAG systems, dense similarity becomes l...
By Nabaraj Subedi, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar
arXiv:2606. 01542v1 Announce Type: cross Abstract: Chunked-document retrieval is a common component of retrieval-augmented generation (RAG) systems.
By Nataraj Agaram Sundar, Tejas Morabia