arXiv Machine Learning
Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
Aug 28
Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG
The paper investigates whether incorporating an evidence-support signal into retrieval evaluation for retrieval‑augmented generation (RAG) improves downstream decision‑making. Across multiple benchmarks and a TREC RAG 2025 setting, the evidence signal alters retriever rankings but its benefits vary: it does not consistently enhance retriever training, its usefulness for system selection depends on generator instructions, and it does not reliably predict answer quality on unseen topics. Human filtering of evidence‑rich passages preserves useful content, yet evaluators disagree on whether this improves final answers, indicating that evidence‑aware evaluation alone does not guarantee better downstream outcomes.
arXiv AI
Jul 29
Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs
arXiv:2607. 25600v1 Announce Type: cross Abstract: Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation.
arXiv AI
Aug 25
LLM-Specific Utility for Retrieval-Augmented Generation
The paper introduces the concept of LLM‑specific utility, defining it as the performance gain a target large language model (LLM) achieves when provided with a passage compared to answering without evidence. A benchmark of utilitarian passages is built for four LLMs (Qwen3‑8B/14B/32B and Llama 3.1‑8B) across three QA datasets, revealing that each model benefits most from its own tailored evidence and that evidence optimized for other models is consistently suboptimal. The authors also create SpecUBench, a benchmark for LLM‑specific utility judgment, and show that current utility‑aware retrieval methods largely capture model‑agnostic usefulness, struggling to estimate LLM‑specific utility. "whyItMatters":"The study demonstrates that retrieval‑augmented generation must consider model‑specific evidence selection to truly improve LLM performance, highlighting a gap in existing utility‑aware methods."
Hugging Face Trending Papers
Jul 28
Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs
Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation. We investigate whether verbalized confidence from black-box language models can serve as an actionable signal for retrieval routing.
arXiv AI
1d ago
Regime Boundary Alignment for Evidence-Gated Question Answering
arXiv:2609.37491v1 Announce Type: cross Abstract: Retrieval-augmented language models are expected to answer from the retrieved evidence, but in practice they often keep answering when that evidence...
arXiv AI
Jun 30
AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering
arXiv:2606. 29090v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) has become the standard way to ground large language models in external knowledge, yet most systems retrieve a fixed number of passages for every question regardless of its difficulty.