Conformal Factuality Control for Multi-Hop Retrieval-Augmented Generation
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2610.34872v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) can improve factual grounding without guaranteeing that every generated claim is supported. Previous work has st...
Retrieval-augmented generation (RAG) can improve factual grounding without guaranteeing that every generated claim is supported. Previous work has studied claim-level conformal factuality in single-st...
Retrieval‑augmented generation systems attach citations to claims, but these citations often misstate provenance. By replacing facts in retrieved passages with false variants and re‑querying the model, the study finds that 1.2% to 15.7% of cited claims actually come from the model’s parametric memory rather than the retrieved context, with larger models showing higher rates. Standard natural‑language‑inference verification fails to detect this, and even the best attribution methods only distinguish provenance by comparing the model’s output distribution with and without context, leaving the problem unresolved.
Retrieval‑augmented generation systems attach citations to claims, but these citations often misstate provenance. By substituting facts in retrieved passages with false variants, the study shows that 1.2% to 15.7% of cited claims actually come from the model’s parametric memory rather than the retrieved context, with larger models exhibiting higher rates. Standard natural‑language‑inference verification fails to detect this, and even the best attribution method only distinguishes provenance by comparing the model’s output distribution with and without context, leaving accurate provenance detection unresolved.
arXiv:2605. 03534v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds answers in retrieved passages, yet relevance does not guarantee sufficiency: a topical passage may still fail to justify the answer.
The paper investigates why large language models (LLMs) miss evidence in multi‑hop retrieval‑augmented generation (RAG). It introduces ExecutionRAG, a training‑free register that tracks open sub‑questions and their supporting evidence, allowing the authors to distinguish between true retrieval failures and search drift. Using this method on 1,000 MuSiQue questions, the study shows a significant reduction in missed evidence but also reveals that other error types, such as premature resolutions and abstentions, increase, shifting the bottleneck from search guidance to evidence verification.