arXiv Computation and Language

System One models Are Not Better Rerankers

arXiv Machine Learning
Oct 28

Can System-1-Style Decision Models Replace Dedicated Rerankers in RAG? A Systems Evaluation

The paper investigates whether System‑1‑style decision models can replace dedicated rerankers in retrieval‑augmented generation (RAG) systems. After screening 21 open‑weight models on 11 benchmarks, the authors compare a promising decision model with a dedicated reranker on identical candidate lists. While limiting candidate text length improves reliability and makes some configurations competitive, a tuned reranker remains faster and uses less GPU memory, indicating that replacing a reranker requires balancing quality, latency, and memory.

By Qiaomu Li, Ying Xie
arXiv Machine Learning
Aug 28

Equal Ranking Quality, Different Decisions: Training Order-Consistent LLM Scorers

The paper investigates how the order of candidate documents in a prompt affects the decisions made by large‑language‑model (LLM) scorers, even when their ranking quality is similar. It shows that five scorers with only a 0.010 nDCG@10 difference can produce retained‑set overlaps as low as 0.66–0.84, and that existing rerankers still exhibit significant order dependence. The authors propose Order‑Consistency SFT (OC‑SFT), a training method that reduces this dependence, maintaining ranking quality while improving decision stability across multiple tasks.

By Markus Frohmann, Mahdiyar Alavi, Elizabeth Lingg, Navid Rekabsaz
arXiv AI
5d ago

Predicting What Answer Engines Cite: Open Decision Models, Rerankers and a Distilled 4B Student on Ten Engines under Controlled Retrieval

The paper introduces a new scorer for generative answer engines that ranks candidate pages by the likelihood of being cited. It evaluates pretrained rerankers, a hosted typed decision model (Jev), eight open decision models, and a distilled 4B student on GEO‑Bench queries answered by ten engines. The distilled student, fine‑tuned with low‑rank adapters, achieves comparable performance to larger models and the authors release the model, benchmark, predictions, and code.

By Elisha Bajemon, Andre-Louis Rochet
Hugging Face Trending Papers
6d ago

Glance: The Generator Is Its Own Reranker

Glance is a retrieval‑augmented generation approach that eliminates the need for separate reranker models by adding a lightweight decision adapter and pointer head to a single 9B backbone. It scores candidate passages directly from a shared prefill, requiring only about 0.1 GiB of extra memory per decision skill. In fine‑tuning on MS MARCO, Glance outperforms the bge‑reranker‑v2‑m3 on SciFact and NFCorpus, and improves answer F1 on HotpotQA by 0.045, while scaling to 200 candidates it surpasses cross‑encoder reranking by 0.058 nDCG@10.

arXiv AI
Oct 16

Prism-1: Position- and Rival-Independent Scoring for System One Models

Prism-1 introduces Position- and Rival-Independent Scoring for System One models, ensuring that option ordering and set changes do not affect decision probabilities. It achieves this by reading each option in its own block at shared positions, preserving softmax independence across options. Experiments on Qwen3-8B and Gemma 4 12B show Prism-1 maintains baseline accuracy while eliminating sensitivity to shuffling and irrelevant options.

By Yao Long Teng, Jianguo Huang, Yanchen Deng, Jiayi Cai, Bo An
arXiv AI
Aug 13

CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generation

arXiv:2605. 04495v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) relies on evidence ranking to determine what information is exposed to the generator, yet existing retrieval and reranking methods primarily estimate query--document relevance.

By Zhipeng Song, Yizhi Zhou, Xiangyu Kong, Jiulong Jiao, Xuezhou Ye, Chunqi Gao, Xueqing Shi, Yu Wang, Yuhang Zhou, Heng Qi
arXiv AI
Oct 16

Verdict: Encode Once, Decide Many

Verdict-MQ is an open encoder that processes a single piece of state once and then answers multiple typed questions about that state, avoiding separate passes for each question. By using a structured attention mask, it ensures the state encoding is independent of question order and prevents questions from influencing one another. On the typed‑decisions benchmark, Verdict‑MQ outperforms a larger specialist model, achieving higher accuracy and better calibration, while on other benchmarks it shows mixed results compared to fine‑tuned baselines.

By Avdhesh Singh Chouhan, Jack Perry
arXiv AI
Jul 8

InfluMatch: Frontier-Quality KOL Search at 4B-Model Cost

arXiv:2607. 05968v1 Announce Type: cross Abstract: Matching influencers (KOLs) to free-form, multi-part Thai marketing criteria is today served either by keyword search over structured profiles, which misses semantic fit, or by prompting frontier LLMs over every candidate, which is accurate but slow and expensive.

By Krittanon Kaewtawee, Petmongkon Pornpichitsuwan, Natchaya Temyingyong, Nutnicha Laplamoon, Wachiravit Modecrua, Krittin Pachtrachai, Touchapon Kraisingkorn