The paper evaluates how robust large language models (LLMs) are when using retrieval‑augmented generation (RAG) in practical settings. It investigates whether RAG always outperforms non‑RAG approaches, whether adding more retrieved documents helps, and whether the order of documents matters, using a benchmark of 1,891 samples across five datasets and three task categories. Experiments with 11 LLMs show generally high retrieval robustness, but performance varies by task and prompting strategy, indicating that adopting RAG should be considered on a case‑by‑case basis.
arXiv:2606. 28337v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems are often evaluated using final answer accuracy, even though their failures can originate from preprocessing, retrieval, context packing, or generation.
The paper introduces LLM-QL, a dense retrieval model that harnesses large language models (LLMs) by maximizing query likelihood (QL) as an auxiliary task. It incorporates an Attention Block to limit predictive token attention to document tokens before the ending token and a Document Corruption component that masks parts of the document during prediction. Experiments on MS MARCO and BEIR datasets show that LLM-QL outperforms other LLM-based retrievers, and detailed analyses confirm the effectiveness of its components.
Retrieval-Augmented Generation (RAG) has become the standard way to ground large language models in external knowledge, yet most systems retrieve a fixed number of passages for every question regardless of its difficulty. This wastes computation on easy questions, starves hard ones, and gives no signal for when a generated answer can be trusted.
Q2D-Web is a new large‑scale benchmark for agentic Retrieval‑Augmented Generation (RAG) systems, featuring a 190 million‑document web corpus and 70 k machine‑reformulated search queries in ten languages. It supplies three sets of relevance judgments—agent citations, production rankings, and a combined set enriched with LLM‑based labels—to evaluate first‑stage retrievers. Experiments on 13 retrievers show consistent ranking across judgment sets but significant variation across domains, languages, and query types, and demonstrate that a carefully sampled sub‑corpus can approximate full‑corpus evaluation with minimal loss in Recall@1000.
The paper introduces the concept of LLM‑specific utility, defining it as the performance gain a target large language model (LLM) achieves when provided with a passage compared to answering without evidence. A benchmark of utilitarian passages is built for four LLMs (Qwen3‑8B/14B/32B and Llama 3.1‑8B) across three QA datasets, revealing that each model benefits most from its own tailored evidence and that evidence optimized for other models is consistently suboptimal. The authors also create SpecUBench, a benchmark for LLM‑specific utility judgment, and show that current utility‑aware retrieval methods largely capture model‑agnostic usefulness, struggling to estimate LLM‑specific utility. "whyItMatters":"The study demonstrates that retrieval‑augmented generation must consider model‑specific evidence selection to truly improve LLM performance, highlighting a gap in existing utility‑aware methods."
arXiv:2606. 29090v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) has become the standard way to ground large language models in external knowledge, yet most systems retrieve a fixed number of passages for every question regardless of its difficulty.
The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.
arXiv:2509.20377v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) has significantly improved the performance of large language models (LLMs) on knowledge-intensive tasks...
The paper introduces STAIR, a retrieval system that uses a document’s Table of Contents to guide large language models in accessing global structure, thereby reducing hallucinations in Retrieval Augmented Generation. Experiments with a fine‑tuned Differentiable Search Index show that ToC‑based retrieval yields a low hallucination rate (<0.05%) and improves Recall@1 to 82.6% on the newly released SearchTome benchmark, outperforming baselines like BM25, DPR, and Mistral. The authors also release SearchTome, a diverse dataset of 18 books across six domains, to encourage further research in ToC‑based retrieval.
The paper introduces DRAG, a query‑adaptive framework that jointly selects retriever and generator configurations for Retrieval‑Augmented Generation (RAG) systems. Two variants are presented: DRAG_QPP, a training‑free routing method using Query Performance Prediction and perplexity signals, and DRAG_SFT, a supervised approach that fine‑tunes an LLM to predict configurations. Experiments on three LLM families and four QA benchmarks show that DRAG_QPP matches strong static baselines while cutting inference latency, and DRAG_SFT consistently outperforms both static and training‑free adaptive baselines, demonstrating a better effectiveness‑efficiency trade‑off.
arXiv:2607. 25600v1 Announce Type: cross Abstract: Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation.
arXiv:2509. 16780v3 Announce Type: replace-cross Abstract: Large language models (LLMs) show promise as educational aids but often lack alignment with specific course materials.