arXiv Computation and Language
System One models Are Not Better Rerankers
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Oct 28
Can System-1-Style Decision Models Replace Dedicated Rerankers in RAG? A Systems Evaluation
The paper investigates whether System‑1‑style decision models can replace dedicated rerankers in retrieval‑augmented generation (RAG) systems. After screening 21 open‑weight models on 11 benchmarks, the authors compare a promising decision model with a dedicated reranker on identical candidate lists. While limiting candidate text length improves reliability and makes some configurations competitive, a tuned reranker remains faster and uses less GPU memory, indicating that replacing a reranker requires balancing quality, latency, and memory.
Hugging Face Trending Papers
Oct 26
Can System-1-Style Decision Models Replace Dedicated Rerankers in RAG? A Systems Evaluation
Retrieval-augmented generation (RAG) systems often use a dedicated reranker to choose which retrieved candidates reach the language model. Reranking improves relevance but adds latency, so teams look...
arXiv Machine Learning
Aug 28
Equal Ranking Quality, Different Decisions: Training Order-Consistent LLM Scorers
The paper investigates how the order of candidate documents in a prompt affects the decisions made by large‑language‑model (LLM) scorers, even when their ranking quality is similar. It shows that five scorers with only a 0.010 nDCG@10 difference can produce retained‑set overlaps as low as 0.66–0.84, and that existing rerankers still exhibit significant order dependence. The authors propose Order‑Consistency SFT (OC‑SFT), a training method that reduces this dependence, maintaining ranking quality while improving decision stability across multiple tasks.
arXiv Computation and Language
1d ago
Every Clause Counts: Instruction-Following Reranking by Multiplying Small, Answerable Questions
arXiv:2611.03179v1 Announce Type: cross Abstract: People often attach instructions to their search queries, asking for pages written for beginners, documents in a certain language or official sources...
arXiv AI
5d ago
Predicting What Answer Engines Cite: Open Decision Models, Rerankers and a Distilled 4B Student on Ten Engines under Controlled Retrieval
The paper introduces a new scorer for generative answer engines that ranks candidate pages by the likelihood of being cited. It evaluates pretrained rerankers, a hosted typed decision model (Jev), eight open decision models, and a distilled 4B student on GEO‑Bench queries answered by ten engines. The distilled student, fine‑tuned with low‑rank adapters, achieves comparable performance to larger models and the authors release the model, benchmark, predictions, and code.
Hugging Face Trending Papers
6d ago
Glance: The Generator Is Its Own Reranker
Glance is a retrieval‑augmented generation approach that eliminates the need for separate reranker models by adding a lightweight decision adapter and pointer head to a single 9B backbone. It scores candidate passages directly from a shared prefill, requiring only about 0.1 GiB of extra memory per decision skill. In fine‑tuning on MS MARCO, Glance outperforms the bge‑reranker‑v2‑m3 on SciFact and NFCorpus, and improves answer F1 on HotpotQA by 0.045, while scaling to 200 candidates it surpasses cross‑encoder reranking by 0.058 nDCG@10.