Hugging Face Blog

Introducing RTEB: A New Standard for Retrieval Evaluation

arXiv Computation and Language
Sep 15

TF-IDF and BM25 Are Exact KL Divergences

arXiv:2609.14016v1 Announce Type: cross Abstract: TF-IDF and BM25 are two of the most widely used methods for scoring query-document relevance, yet neither has a standard probabilistic derivation tha...

By Ivan Silajev
arXiv Computation and Language
Sep 3

Incremental Pooled LLM Evaluation for Cost-Effective Retrieval Model Selection

The paper proposes an incremental pooled LLM evaluation method for selecting retrieval models in production Retrieval-Augmented Generation (RAG) systems. By having a language model judge the union of documents retrieved by current candidates and expanding the pool only with new documents from added systems, the approach reuses judgments across all systems. Experiments on four benchmarks and a financial news QA deployment show strong correlation with gold-standard rankings, high preservation of pairwise orderings, and significant cost savings—up to 4.9× lower evaluation cost and 65–80% judgment reuse.

By Max Nelson, Hanoz Bhathena, Aviral Joshi, Saket Sharma
arXiv Computation and Language
Aug 25

L\"etzCross: A Cross-Lingual Page-Level Benchmark for Multimodal Retrieval over Luxembourgish Documents

arXiv:2608.21714v1 Announce Type: new Abstract: Recent page-image retrievers such as ColPali have improved retrieval over visually rich documents, yet little is known about how they behave in cross-l...

By Omar El Bachyr, Fred Philippy, Laura Maria Bernardy, Saad Ezzini, Jacques Klein, Tegawende Bissyande
arXiv Computation and Language
Aug 27

Align Then Adapt: Label-Efficient Adapter Learning for Asymmetric Dense Retrieval

The paper introduces Efficient Retrieval Adapter (ERA), a query‑side adapter framework that enables dense retrieval systems to adapt to asymmetric query–document scenarios without re‑indexing. ERA first aligns the embedding spaces of a powerful query embedder and a lightweight document embedder using unlabeled documents, then fine‑tunes the aligned query representation with a small set of labeled query‑document pairs. In experiments on 126 MAIR retrieval tasks across six domains, ERA boosts average nDCG@10 by up to 8.2 points in symmetric settings and over 12 points in asymmetric settings while requiring far fewer labels than fully supervised adapter training.

By Seiji Maekawa, Moin Aminnaseri, Pouya Pezeshkpour, Estevam Hruschka
arXiv Computation and Language
Aug 31

Search, Inspect, Fetch: Exploiting Structure-Aware Boolean Retrieval for Deep-Search Agents

The paper introduces Sieve, a search‑inspect‑fetch framework that leverages a Boolean Query Language (BQL) to target specific webpage fields, rank candidates, present structure‑rich result cards, and fetch only selected sections. Compared to traditional Search‑Visit agents, Sieve achieves higher accuracy across three QA collections while reducing token usage by 20.7–50.6%. Boolean filtering consistently improves performance for all tested rankers and remains effective across different retrievers and agent backbones.

By Shuai Wang, Haodong Chen, Yu Yin, Shengyao Zhuang, Bevan Koopman, Guido Zuccon