arXiv Computation and Language August 31, 2026 By Syed Mahbubul Huq, Christopher Child, Tillman Weyde, Pranava Madhyastha

Knowing Before Answering: Decoding Language Models for Reliable RAG

Read the original on arXiv Computation and Language →

The paper introduces a method for determining whether retrieval-augmented generation (RAG) systems have sufficient, insufficient, or conflicting evidence to answer a question. By training a lightweight linear classifier on hidden activations and attention-derived features from 16 language models, the authors demonstrate that these internal signals reliably predict the adequacy of retrieved documents, outperforming prompting-based baselines and specialized RAG models. Analysis shows that middle-layer hidden states carry the most informative signals for this triage task.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Jun 30

AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering

arXiv:2606. 29090v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) has become the standard way to ground large language models in external knowledge, yet most systems retrieve a fixed number of passages for every question regardless of its difficulty.

By Ansh Kamthan

llms rag nlp

arXiv Machine Learning

Jun 2

Grounded Decoding: Retrieval-Anchored Probability Fusion for Faithful RAG

arXiv:2606. 00432v1 Announce Type: new Abstract: As retrieval-augmented generation (RAG) systems scale, it becomes increasingly challenging to ensure faithful grounding in external evidence.

By Ibne Farabi Shihab, Fariya Afrin, Sanjeda Akter, Anuj Sharma

llms rag

arXiv Computation and Language

Aug 28

Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG

The paper investigates whether incorporating an evidence-support signal into retrieval evaluation for retrieval‑augmented generation (RAG) improves downstream decision‑making. Across multiple benchmarks and a TREC RAG 2025 setting, the evidence signal alters retriever rankings but its benefits vary: it does not consistently enhance retriever training, its usefulness for system selection depends on generator instructions, and it does not reliably predict answer quality on unseen topics. Human filtering of evidence‑rich passages preserves useful content, yet evaluators disagree on whether this improves final answers, indicating that evidence‑aware evaluation alone does not guarantee better downstream outcomes.

By Utshab Kumar Ghosh, Debayan Mukhopadhyay, Shubham Chatterjee

rag benchmarks safety

Hugging Face Trending Papers

Jun 27

AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering

Retrieval-Augmented Generation (RAG) has become the standard way to ground large language models in external knowledge, yet most systems retrieve a fixed number of passages for every question regardless of its difficulty. This wastes computation on easy questions, starves hard ones, and gives no signal for when a generated answer can be trusted.

llms rag nlp

arXiv AI

Aug 24

SKILL-RAG: Self-Knowledge Induced Learning and Filtering for Retrieval-Augmented Generation

arXiv:2509.20377v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) has significantly improved the performance of large language models (LLMs) on knowledge-intensive tasks...

By Tomoaki Isoda

llms rag nlp reinforcement-learning benchmarks safety

arXiv AI

Aug 11

Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Models

arXiv:2512. 11614v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) relies on retrieved context to guide large language models (LLM), yet treats the retrieval as a heuristic rather than verifiable evidence -- leading to unsupported answers, hallucinations, and reliance on spurious context.

By Bj\"orn Deiseroth, Max Henning H\"oth, Kristian Kersting, Letitia Parcalabescu

llms rag agents fine-tuning benchmarks safety

One email a morning, machine-written

Related stories

AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering

Grounded Decoding: Retrieval-Anchored Probability Fusion for Faithful RAG

Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG

AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering

SKILL-RAG: Self-Knowledge Induced Learning and Filtering for Retrieval-Augmented Generation

Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Models