← Back to all news
arXiv Computation and Language By Korbinian Randl, Guido Rocchietti, Aron Henriksson, Ziawasch Abedjan, Tony Lindgren, John Pavlopoulos

Quantifying Retriever-Generator Alignment in RAG with Local Explanations

Read the original on arXiv Computation and Language →

The paper introduces RAG-E, an explainability framework for Retrieval-Augmented Generation (RAG) systems that quantifies how well a language model’s document usage aligns with the retriever’s rankings. It adapts Integrated Gradients for retriever analysis, uses a Monte Carlo‑stabilized Shapley Value approximation for generator attribution, and proposes the Weighted Alignment between Retriever and Generator (WARG) metric. Experiments on PopQA, QAMPARI, and TREC CAST show that generators often ignore top‑ranked documents, and WARG outperforms Pearson and Spearman correlations in measuring alignment, offering a practical tool for auditing RAG performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Oct 5

Evaluating the Retrieval Robustness of Large Language Models

The paper evaluates how robust large language models (LLMs) are when using retrieval‑augmented generation (RAG) in practical settings. It investigates whether RAG always outperforms non‑RAG approaches, whether adding more retrieved documents helps, and whether the order of documents matters, using a benchmark of 1,891 samples across five datasets and three task categories. Experiments with 11 LLMs show generally high retrieval robustness, but performance varies by task and prompting strategy, indicating that adopting RAG should be considered on a case‑by‑case basis.

By Shuyang Cao, Karthik Radhakrishnan, David Rosenberg, Steven Lu, Pengxiang Cheng, Lu Wang, Shiyue Zhang
llmsragbenchmarks
More like this →
arXiv AI
Jul 1

RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora

arXiv:2604. 19047v2 Announce Type: replace-cross Abstract: Existing QA benchmarks typically assume distinct documents with minimal overlap, yet real-world retrieval-augmented generation (RAG) systems operate on corpora such as financial reports, legal codes, and patents, where information is highly redundant and documents exhibit strong inter-document similarity.

By Hanjun Cho, Jay-Yoon Lee
llmsragbenchmarks
More like this →
arXiv Computation and Language
5d ago

The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering

arXiv:2610.26951v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunk...

By Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum, Zhuohan Xie, Preslav Nakov, Veselin Stoyanov
llmsragnlpbenchmarks
More like this →