arXiv Machine Learning

SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation

arXiv:2605. 03534v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds answers in retrieved passages, yet relevance does not guarantee sufficiency: a topical passage may still fail to justify the answer.

arXiv Machine Learning

Aug 11

What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventions

arXiv:2608. 08944v1 Announce Type: cross Abstract: A failed retrieval-augmented generation (RAG) answer can be consistent with several unseen responses to evidence repair.

By Wenzhang Du

rag benchmarks

arXiv AI

Jul 21

Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict

arXiv:2605. 14473v4 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) is usually evaluated by whether the final answer is correct.

By Yihang Chen, Pin Qian, Su Wang, Sipeng Zhang, Huan Xu, Shuhuai Lin, Xinpeng Wei

llms rag safety

arXiv Machine Learning

6d ago

LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence

arXiv:2608. 11922v1 Announce Type: cross Abstract: Predictive-distribution entropy makes a strong selection rule in retrieval-augmented question answering: across five QA benchmarks, keeping the candidate answer that a frozen respondent LLM produces with the lowest answer-token entropy lifts mean answer $F_1$ from 0.

By Po-Jen Ko, Che-Cheng Wu, Hung-Chun Hsu, Li-Yang Chang, Chuan-Ju Wang

llms rag nlp reinforcement-learning benchmarks

arXiv AI

Jun 6

Answer Presence Drives RAG Rewriting Gains

arXiv:2606. 05633v1 Announce Type: new Abstract: Retrieval-augmented QA pipelines often route retrieved passages through an LLM \emph{rewriter} before a smaller reader, lifting F1 by tens of points on multi-hop benchmarks; this gain is typically credited to improved evidence quality.

By Yuejie Li, Yueying Hua, Ke Yang, Li Zhang, Yueping He, Yueping He, Ruiqi Li, Bolin Chen, Tao Wang, Bowen Li, Chengjun Mao

llms rag benchmarks

arXiv Machine Learning

Jun 24

Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

arXiv:2606. 23915v1 Announce Type: cross Abstract: Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable.

By Tianyu Ding, Aditya Nannapaneni, Juan Pablo De la Cruz Weinstein

llms rag

arXiv AI

Jul 22

Calibrated Selective Fact-Checking via Evidence Chain Evaluation

arXiv:2607. 18240v1 Announce Type: new Abstract: Large language models (LLMs) can achieve strong fact-checking accuracy, yet forced binary decisions conceal a critical reliability problem: systems may issue confident verdicts even when supporting evidence is weak, sparse, or internally inconsistent.

By Dekun Yang

llms agents safety

arXiv AI

Jul 7

Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

arXiv:2607. 04223v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why.

By Mohamed Aly Bouke

rag nlp benchmarks safety

arXiv Machine Learning

Jul 7

Coverage-Controlled Preference Mining from Noisy Claim Verification for Evidence-Grounded Generation

arXiv:2603. 10494v2 Announce Type: replace-cross Abstract: Evidence-grounded generation produces summaries whose claims should be supported by supplied evidence, but claim-level verifiers provide noisy feedback and can reward models that simply say less.

By Weixin Liu, Congning Ni, Qingyuan Song, Susannah L. Rose, Murat Kantarcioglu, Bradley A. Malin, Zhijun Yin

llms rag nlp reinforcement-learning

Hugging Face Trending Papers

1d ago

Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees

Using LLMs as judges has become standard practice for evaluating model outputs at scale. This is particularly common for subjective, open-ended tasks such as assessing helpfulness or alignment, where no single reference answer exists.

llms rag benchmarks safety

arXiv AI

Jun 2

Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking

arXiv:2606. 01120v1 Announce Type: new Abstract: In RAG-based fact-checking, LLMs are increasingly used as verifiers to check given claims against retrieved evidence.

By Yuxi Sun, Wenbo Shang, Wei Gao, Xin Huang, Jing Ma

llms rag benchmarks

arXiv Machine Learning

Jul 30

Same Evidence, Different Target: Decoding How Diagnostic Evidence Bears on Causal Questions from Language-Model States

arXiv:2607. 26929v1 Announce Type: cross Abstract: The same diagnostic result can support or challenge one causal claim yet fail to address another when the claims concern different populations, outcomes, estimands, pathways, or identifying assumptions.

By Weiyi Kong, Zhuoran Li

llms benchmarks

arXiv AI

Jul 2

Theoria: Rewrite-Acceptability Verification over Informal Reasoning States

arXiv:2607. 01223v1 Announce Type: new Abstract: When should an AI system's answer be trusted?

By Ben Slivinski, Michael Saldivar

llms safety

Related stories