The paper investigates whether System‑1‑style decision models can replace dedicated rerankers in retrieval‑augmented generation (RAG) systems. After screening 21 open‑weight models on 11 benchmarks, the authors compare a promising decision model with a dedicated reranker on identical candidate lists. While limiting candidate text length improves reliability and makes some configurations competitive, a tuned reranker remains faster and uses less GPU memory, indicating that replacing a reranker requires balancing quality, latency, and memory.
By Qiaomu Li, Ying Xie
arXiv:2610.32318v1 Announce Type: new
Abstract: System One models, or decision models, answer a question about an input with a calibrated probability instead of generated text. Within weeks of the fi...
By Michael Dinzinger, Kanishka Ghosh Dastidar, Michael Granitzer
Glance is a retrieval‑augmented generation approach that eliminates the need for separate reranker models by adding a lightweight decision adapter and pointer head to a single 9B backbone. It scores candidate passages directly from a shared prefill, requiring only about 0.1 GiB of extra memory per decision skill. In fine‑tuning on MS MARCO, Glance outperforms the bge‑reranker‑v2‑m3 on SciFact and NFCorpus, and improves answer F1 on HotpotQA by 0.045, while scaling to 200 candidates it surpasses cross‑encoder reranking by 0.058 nDCG@10.
arXiv:2609.40241v1 Announce Type: cross
Abstract: Large language models (LLMs) have shown promise for recommendation reranking, but their use introduces an important tradeoff between recommendation q...
By Hanjia Lyu, Yinglong Xia
The study evaluates large language models (LLMs) as rerankers in conversational movie recommendation, comparing proprietary, open-weight, and fine-tuned LLMs against collaborative-filtering and sequential baselines on the ReDial benchmark. Results show that the best proprietary LLM achieves an NDCG@10 of 0.1497 with a shared semantic candidate pool, outperforming non-LLM baselines, while open-weight LLMs do not surpass a tuned shallow autoencoder under the same protocol. The analysis also highlights that reranker performance is highly sensitive to candidate generation, pool size, scoring policy, and decoding temperature, suggesting these factors should be reported as standard evaluation fields.
By Ante Kapetanovic, Tomislav Duricic, Andro Mercep, Emanuel Lacic
arXiv:2602.18613v2 Announce Type: replace
Abstract: Standard reranking evaluations study how a reranker orders candidates returned by an upstream retriever. This setup couples ranking behavior with r...
By Baris Arat, Emre Sefer
Re$^3$Rank is a listwise reranker that uses reasoning-conditioned retrieval heads (RRHeads). It generates a reasoning trace for each candidate, then scores candidates by using the trace tokens as attention queries for selected RRHeads, producing continuous relevance scores. A straight‑through estimator allows joint training of the generator and scorer, and on nine benchmarks the 4B model outperforms 11 baseline rerankers by an average of 3.11 NDCG@10 points, while the 0.6B model beats rerankers with up to 32B parameters.
By Ruiyi Yan, Zhuoyuan Mao, Yugo Murawaki, Yiwen Guo
DecisionRank introduces an on-policy distillation (OPD) method that trains decision models to replicate a generative LLM’s ranking knowledge by converting the student’s document scores into a ranked sequence and using a teacher to provide conditional feedback. The approach combines item‑space OPD with offline ranking supervision, achieving state‑of‑the‑art results on retrieval benchmarks such as TREC DL, BEIR, and BRIGHT. Even with a 1.7B parameter model, DecisionRank outperforms its 27B generative teacher on TREC DL19–23, and at 8B it surpasses the generative listwise counterpart while running 24.5× faster on TREC DL and 16.7× faster on BRIGHT.
By Yanran Wu, Dian Yu, Chongyang Bai, Jieyi Zhang, Rui Liu, Tianyi Chen, Shike Mei, Hang Cui, Frank Shyu, Sandeep Pandey, Luke Simon, Chunwei Liu, Xi Liu
The paper introduces a new scorer for generative answer engines that ranks candidate pages by the likelihood of being cited. It evaluates pretrained rerankers, a hosted typed decision model (Jev), eight open decision models, and a distilled 4B student on GEO‑Bench queries answered by ten engines. The distilled student, fine‑tuned with low‑rank adapters, achieves comparable performance to larger models and the authors release the model, benchmark, predictions, and code.
By Elisha Bajemon, Andre-Louis Rochet
Anchor-based pointwise LLM reranking scores each candidate against a shared reference passage to recover cross-document context at pointwise cost. We study when this actually helps, using GCCP/PAGC as a representative method.
arXiv:2606. 03650v1 Announce Type: cross Abstract: Choosing or ranking language models for a specific application is hardest when no task-specific labeled data exists, and standard public benchmarks cannot be trusted, their items having likely leaked into pretraining, so scores reflect memorization rather than fitness.
By Alexander Apartsin, Yehudit Aperstein
The paper proposes an incremental pooled LLM evaluation method for selecting retrieval models in production Retrieval-Augmented Generation (RAG) systems. By having a language model judge the union of documents retrieved by current candidates and expanding the pool only with new documents from added systems, the approach reuses judgments across all systems. Experiments on four benchmarks and a financial news QA deployment show strong correlation with gold-standard rankings, high preservation of pairwise orderings, and significant cost savings—up to 4.9× lower evaluation cost and 65–80% judgment reuse.
By Max Nelson, Hanoz Bhathena, Aviral Joshi, Saket Sharma