The paper examines LLM-based recommendation rerankers that are often evaluated under an oracle protocol, which guarantees the ground-truth item is present in the scored set. Across Amazon datasets, this protocol overestimates realistic NDCG@10 by 92–95% because realistic retrieval only covers 2–19% of relevant items at K=100, creating a recall ceiling that limits any closed-candidate reranker's top‑k NDCG. The authors find that various optimisation strategies—including prompt engineering, model scaling, sequential models, supervised neural rerankers, LoRA fine‑tuning, hybrid retrieval, score‑aware prompting, and LLM+CF fusion—do not significantly improve over a collaborative‑filtering baseline under realistic retrieval, and they propose a Recall‑Aware Evaluation Protocol (RAEP) to better assess rerankers in low‑recall regimes.
By Zhaohui Wang
The study evaluates large language models (LLMs) as rerankers in conversational movie recommendation, comparing proprietary, open-weight, and fine-tuned LLMs against collaborative-filtering and sequential baselines on the ReDial benchmark. Results show that the best proprietary LLM achieves an NDCG@10 of 0.1497 with a shared semantic candidate pool, outperforming non-LLM baselines, while open-weight LLMs do not surpass a tuned shallow autoencoder under the same protocol. The analysis also highlights that reranker performance is highly sensitive to candidate generation, pool size, scoring policy, and decoding temperature, suggesting these factors should be reported as standard evaluation fields.
By Ante Kapetanovic, Tomislav Duricic, Andro Mercep, Emanuel Lacic
arXiv:2608.24079v1 Announce Type: cross
Abstract: A shared search-and-recommendation index must score new items from features alone because search has no exploration slot. In a public log covering bo...
By Theodore Rogers, Joe Standerfer, Dmitrii Timoshenko, Haoxue Li, Zuhaib Akhtar, Soyoung Yang
arXiv:2607. 09877v1 Announce Type: new Abstract: Vacation rental marketplaces face a structural imbalance on the supply side: a small fraction of properties receive most user interactions, while the long tail of new, niche, and seasonal listings generates too little behavioral signal for collaborative filtering to serve effectively.
By Syed Mohammed Arshad Zaidi, Eric Rincon, Shayan Hassantabar
arXiv:2602.18613v2 Announce Type: replace
Abstract: Standard reranking evaluations study how a reranker orders candidates returned by an upstream retriever. This setup couples ranking behavior with r...
By Baris Arat, Emre Sefer
The paper reports the ABAI submission to COLIEE 2026 Task 1, a case law retrieval challenge that suppresses cited passages, and details a four‑stage retrieval pipeline: multi‑view BM25 with reciprocal rank fusion, neural reranking, graph‑based features via a graph attention network, and a LightGBM meta‑learner over 34 features. The best run achieved an F1 score of 0.177 on the official test set, compared to a cross‑validated 0.311, and the authors attribute the gap to a recall ceiling, temporal distribution shift, and threshold miscalibration. A controlled post‑hoc study examined the impact of threshold transfer, decision quality across time, and query similarity, and identified specific remedies—such as BM25 length‑normalisation tuning, event‑triple views, and dense fusion—that improved recall, while other interventions had no effect.
By Minhan Cho, Soyoung Park, Daejin Choi, Jinyoung Han
The paper investigates the trade‑off of using a shared search‑and‑recommendation index that scores new items purely from features, thereby keeping the index open to unseen items. Experiments on public logs show that a feature‑based tower can match warm‑item performance (Recall@20 0.9595 vs 0.9510) and a lexical baseline, while a full‑catalog check is inconclusive. The study also quantifies the cost of this openness on recommendation quality across several baselines, revealing that exact full‑softmax training improves recall but is impractical at catalog scale.
By Theodore Rogers, Joe Standerfer, Dmitrii Timoshenko, Haoxue Li, Zuhaib Akhtar, Soyoung Yang
The paper introduces “PACE”, a training‑free framework that tackles bottlenecks in Retrieval‑Augmented Generation by frontloading evidence and adaptively budgeting reranking. It first reorders candidate documents based on marginal evidence coverage—prioritizing query‑relevant, complementary, and chain‑forming documents—providing a $(1-1/e)$ approximation guarantee. Then it dynamically adjusts the reranking budget according to the relative pressure of the reranker and the language model, improving evidence recall and reducing p95 latency in multi‑hop QA workloads.
By Weibin Cai, Reza Zafarani
arXiv:2605.05726v2 Announce Type: replace
Abstract: As LLM agents are increasingly deployed with large libraries of reusable skills, selecting the right skill for a user request has become a critical...
By Hongcheol Cho, Ryangkyung Kang, Youngeun Kim
arXiv:2605. 04495v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) relies on evidence ranking to determine what information is exposed to the generator, yet existing retrieval and reranking methods primarily estimate query--document relevance.
By Zhipeng Song, Yizhi Zhou, Xiangyu Kong, Jiulong Jiao, Xuezhou Ye, Chunqi Gao, Xueqing Shi, Yu Wang, Yuhang Zhou, Heng Qi
arXiv:2609.37574v1 Announce Type: cross
Abstract: Large Language Models (LLMs) are increasingly used to enrich user queries in information retrieval (IR) so that a standard retriever such as BM25 can...
By Tzu-I Ho, Yung-Yu Shih, Shang-Yu Su, Dongzhe Wang, Yun-Nung Chen
arXiv:2609.13486v1 Announce Type: cross
Abstract: Recent work has shown that fine-tuning decoder-only large language models (LLMs) for retrieval yields strong first-stage retrievers, with effectivene...
By Anubhav Shrestha, Safal Shrestha, Minwu Kim, Torsten Suel, Keith Ross