arXiv AI

Diagnosing the Fact-Grounding Gap in Multi-Hop Question Answering

The paper investigates multi‑hop question answering systems and identifies two distinct failure modes: retrieval failures, where the necessary passage is not retrieved, and extraction failures, where the passage is retrieved but the required fact cannot be extracted—a phenomenon termed the fact‑grounding gap. Across three standard benchmarks, extraction failures account for nearly half of all per‑hop deficiencies and are invisible to standard retrieval metrics, remaining unresolved by retrieval‑only interventions. The study shows that these two bottlenecks require different solutions, a distinction currently missing from evaluation practices.

arXiv Computation and Language

Oct 16

Beyond Retrieval Failure: Understanding Why LLMs Miss Evidence in Multi-Hop RAG

The paper investigates why large language models (LLMs) miss evidence in multi‑hop retrieval‑augmented generation (RAG). It introduces ExecutionRAG, a training‑free register that tracks open sub‑questions and their supporting evidence, allowing the authors to distinguish between true retrieval failures and search drift. Using this method on 1,000 MuSiQue questions, the study shows a significant reduction in missed evidence but also reveals that other error types, such as premature resolutions and abstentions, increase, shifting the bottleneck from search guidance to evidence verification.

By Jiaqi Wei, Wenxuan Huang, Chengdong Xu, Xiang Zhang, Wanli Ouyang

llms rag agents

arXiv Computation and Language

Oct 21

The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering

arXiv:2610.26951v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunk...

By Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum, Zhuohan Xie, Preslav Nakov, Veselin Stoyanov

llms rag nlp benchmarks

arXiv Computation and Language

Oct 15

ConRAG: Consensus-Driven Evidence Flow for Multi-Hop Question Answering

arXiv:2605.28093v3 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) has emerged as a promising paradigm for enhancing large language models (LLMs) on multi-hop question answering...

By Yikai Zhu, Kunfeng Chen, Qihuang Zhong, Juhua Liu, Bo Du

llms rag nlp benchmarks

arXiv AI

Jun 30

A Systems-Level Analysis of Sensitivity, Robustness, and Stability in Retrieval-Augmented Generation

arXiv:2606. 28337v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems are often evaluated using final answer accuracy, even though their failures can originate from preprocessing, retrieval, context packing, or generation.

By Bharath Simha Reddy Muthyam

rag

arXiv AI

Jun 2

RASER: Recoverability-Aware Selective Escalation Router for Multi-Hop Question Answering

arXiv:2606. 02488v1 Announce Type: new Abstract: Multi-hop question-answering systems often use expensive retrieval on every question.

By Yuyang Li, Zihe Yan, Tobias K\"afer

llms rag nlp benchmarks

arXiv Computation and Language

Sep 15

HyperProve: Answer-Guided Hypergraph Expansion for Multi-Hop Question Answering

arXiv:2609.13768v1 Announce Type: new Abstract: Multi-hop question answering often fails when retrieval treats evidence as isolated matches to the original question, since the facts needed to answer...

By An Nguyen Phu, Dung Nguyen Quang, Luu Hieu An, Linh Ngo Van, Trung Le, Thien Huu Nguyen

rag nlp benchmarks safety

arXiv AI

Sep 10

Beyond One-Shot Expansion: Contrastive Evidence Exploration for Multi-Hop Retrieval

arXiv:2609.07050v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) critically depends on retrieving the evidence necessary for effective reasoning. However, this remains particularl...

By JungMin Yun, YoungBin Kim

rag nlp

arXiv Computation and Language

Sep 1

Hi-Q: Hierarchical Evidence-guided Query Refinement for Multi-Hop Question Answering

Hi-Q is a new framework for multi‑hop question answering that refines queries hierarchically based on evidence retrieved from a corpus. At each node it tests whether the current query unit is supported by evidence; if not, the node is expanded using a dependency‑preserving binary operator and verified for semantic coverage. The resulting query tree grows according to corpus support signals, and Hi‑Q achieves state‑of‑the‑art performance on three multi‑hop QA benchmarks, outperforming both iterative retrieval and graph‑based baselines without constructing a corpus‑wide graph.

By Jueun Kim, Sungho Park, Wook-Shin Han

rag nlp benchmarks

arXiv Computation and Language

Aug 31

PRISM: Agentic Retrieval with LLMs for Multi-Hop Question Answering

PRISM is an agentic retrieval framework that uses large language models in a structured loop to improve evidence gathering for multi‑hop question answering. It splits retrieval into three specialized agents—a Question Analyzer, a Selector focused on precision, and an Adder focused on recall—whose iterative interaction yields a compact yet comprehensive evidence set. Experiments on HotpotQA, 2WikiMultiHopQA, MuSiQue, and MultiHopRAG show that PRISM consistently outperforms strong baselines by achieving higher retrieval accuracy and filtering out distracting content.

By Md Mahadi Hasan Nahid, Davood Rafiei

llms agents nlp benchmarks

arXiv AI

Jul 14

STEC: Evidence Compression for Deep Search in Open-domain Multi-Hop QA

arXiv:2607. 10795v1 Announce Type: new Abstract: In open-domain multi-hop question answering (QA), LLM-based search agents offer a promising approach to knowledge-intensive QA by combining retrieval with reasoning.

By Xinkang Li, Rong Jiang, Xin Song, Ye Wang, Yue Han, Changjian Li

llms agents nlp benchmarks

arXiv AI

Aug 26

Omanic: Towards Step-wise Evaluation of Multi-hop Reasoning in Large Language Models

arXiv:2603.16654v3 Announce Type: replace-cross Abstract: Evaluating the reasoning abilities of large language models (LLMs) solely from final answers can obscure failures in intermediate steps, espe...

By Xiaojie Gu, Sherry T. Tong, Aosong Feng, Sophia Simeng Han, Jinghui Lu, Yingjian Chen, Yusuke Iwasawa, Yutaka Matsuo, Chanjun Park, Rex Ying, Irene Li

llms fine-tuning benchmarks

arXiv AI

Sep 30

Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG

arXiv:2609.36700v1 Announce Type: cross Abstract: When conversing with large language models (LLMs), users often begin with a simple question and build towards a multi-hop question through follow-up...

By Pranav Handa, Ariful Azad

llms rag nlp benchmarks

Related stories