arXiv AI
Aug 5
Training Documents Reranker with Search Rubrics for Deep Research Agent
arXiv:2608. 03527v1 Announce Type: cross Abstract: Retrieval systems help deep research agents generate high-quality answers by providing relevant documents.
Beyond Relevance-Centric Retrieval: Rubric-Oriented Document Set Selection and Ranking
The paper introduces SetwiseEvalKit, a benchmark with nine dimensions and 28K rubrics for evaluating document sets, highlighting that current rerankers only achieve up to 45% coverage and struggle with cross‑document coordination. It proposes Rubric4Setwise, a training‑free method that transforms rubric criteria into selection signals, outperforming other methods in downstream generation tasks across both short‑form and long‑form scenarios. The study demonstrates the importance of evaluating and optimizing document sets as a whole rather than independently.
Related stories
arXiv AI
Jul 14
Tool-Adaptive LLM Reranker
arXiv:2607. 10555v1 Announce Type: cross Abstract: Generative Large Language Models (LLMs) have revolutionized information retrieval, yet their strictly parametric nature frequently leads to severe factual hallucinations when confronted with complex queries beyond their epistemic boundaries.
arXiv AI
Jul 3
Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning
arXiv:2604. 02091v2 Announce Type: replace-cross Abstract: Rerankers play a pivotal role in refining retrieval results for Retrieval-Augmented Generation.
arXiv AI
Jun 2
Test-Time Training for Zero-Resource Dense Retrieval Reranking
arXiv:2606. 01070v1 Announce Type: cross Abstract: Dense retrievers excel at first-stage candidate generation but lack effective reranking in zero-resource settings.
arXiv AI
Aug 5
SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG
arXiv:2608. 03860v1 Announce Type: cross Abstract: We introduce SciRet, a compute-aware empirical study of retrieval-augmented generation for scientific question answering over CORD-19.
arXiv AI
Aug 13
CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generation
arXiv:2605. 04495v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) relies on evidence ranking to determine what information is exposed to the generator, yet existing retrieval and reranking methods primarily estimate query--document relevance.
arXiv AI
Aug 10
SAGEO Arena: A Realistic Environment for Evaluating Search-Augmented Generative Engine Optimization
arXiv:2602. 12187v2 Announce Type: replace-cross Abstract: Search-Augmented Generative Engines (SAGE) have emerged as a new paradigm for information access, bridging web-scale retrieval with generative capabilities to deliver synthesized answers.
arXiv Computation and Language
Sep 3
Incremental Pooled LLM Evaluation for Cost-Effective Retrieval Model Selection
The paper proposes an incremental pooled LLM evaluation method for selecting retrieval models in production Retrieval-Augmented Generation (RAG) systems. By having a language model judge the union of documents retrieved by current candidates and expanding the pool only with new documents from added systems, the approach reuses judgments across all systems. Experiments on four benchmarks and a financial news QA deployment show strong correlation with gold-standard rankings, high preservation of pairwise orderings, and significant cost savings—up to 4.9× lower evaluation cost and 65–80% judgment reuse.
Hugging Face Trending Papers
Sep 8
Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems
Q2D-Web is a new large‑scale benchmark for agentic Retrieval‑Augmented Generation (RAG) systems, featuring a 190 million‑document web corpus and 70 k machine‑reformulated search queries in ten languages. It supplies three sets of relevance judgments—agent citations, production rankings, and a combined set enriched with LLM‑based labels—to evaluate first‑stage retrievers. Experiments on 13 retrievers show consistent ranking across judgment sets but significant variation across domains, languages, and query types, and demonstrate that a carefully sampled sub‑corpus can approximate full‑corpus evaluation with minimal loss in Recall@1000.
arXiv AI
Jun 30
Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?
arXiv:2606. 28367v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) is routinely extended with methods meant to improve retrieval: query expansion, hierarchical and cross-document summarization, graph-based expansion, per-query routing, rank fusion, and corrective re-retrieval.
arXiv Machine Learning
Jul 31
(Towards) Scalable Reliable Automated Evaluation with Large Language Models
arXiv:2607. 28282v1 Announce Type: cross Abstract: Evaluating the quality and relevance of textual outputs from Large Language Models (LLMs) remains challenging and resource-intensive.
arXiv Computation and Language
1d ago
TRIO: Task Refinement with Individual Optimization for Retrieval-Augmented Generation
The paper introduces TRIO, a retrieval‑augmented generation framework that uses three agents—query extension, judge, and answer—to iteratively refine queries and improve retrieval and answer quality. It aligns each agent’s objective with rewards derived from answer quality, employs unanimity voting to label expanded queries, and applies dynamic matching‑based rewards with curriculum learning for the answer agent. Experiments on multiple benchmarks show TRIO outperforms state‑of‑the‑art methods.