← Back to all news
arXiv Computation and Language By Emmanouil Georgios Lionis, Debasis Ganguly, Sean MacAvaney

Robustness of IR Models to Collection Growth

Read the original on arXiv Computation and Language →

The paper investigates how adding non‑relevant documents to a collection affects the performance of Information Retrieval (IR) models. It distinguishes between Multi‑Document‑Agnostic (MDA) models, which do not rely on other documents, and Multi‑Document‑Dependent (MDD) models, which do, and evaluates both on two collections with little topic overlap. Results show that all models suffer some performance loss when non‑relevant documents are added, but MDA models outperform MDD models in retrieval tasks, while both perform similarly in reranking.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
Sep 3

Incremental Pooled LLM Evaluation for Cost-Effective Retrieval Model Selection

The paper proposes an incremental pooled LLM evaluation method for selecting retrieval models in production Retrieval-Augmented Generation (RAG) systems. By having a language model judge the union of documents retrieved by current candidates and expanding the pool only with new documents from added systems, the approach reuses judgments across all systems. Experiments on four benchmarks and a financial news QA deployment show strong correlation with gold-standard rankings, high preservation of pairwise orderings, and significant cost savings—up to 4.9× lower evaluation cost and 65–80% judgment reuse.

By Max Nelson, Hanoz Bhathena, Aviral Joshi, Saket Sharma
llmsragbenchmarks
More like this →
arXiv AI
Jul 1

RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora

arXiv:2604. 19047v2 Announce Type: replace-cross Abstract: Existing QA benchmarks typically assume distinct documents with minimal overlap, yet real-world retrieval-augmented generation (RAG) systems operate on corpora such as financial reports, legal codes, and patents, where information is highly redundant and documents exhibit strong inter-document similarity.

By Hanjun Cho, Jay-Yoon Lee
llmsragbenchmarks
More like this →
arXiv Machine Learning
Jun 30

Diagnosing and Mitigating Retrieval Bottlenecks in LLM-Based Cold-Start Recommendation

arXiv:2606. 29947v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as rerankers in recommender systems, with the expectation that semantic understanding will help in cold-start and long-tail regimes.

By Zhe Dong (University of Maine at Presque Isle), Fang Qin (Stanford University), Manish Shah (Independent Researcher), Yicheng Wang (Independent Researcher)
llmsragbenchmarks
More like this →