arXiv Machine Learning
Sep 24

The Recall Ceiling of LLM Recommendation Reranking

The paper examines LLM-based recommendation rerankers that are often evaluated under an oracle protocol, which guarantees the ground-truth item is present in the scored set. Across Amazon datasets, this protocol overestimates realistic NDCG@10 by 92–95% because realistic retrieval only covers 2–19% of relevant items at K=100, creating a recall ceiling that limits any closed-candidate reranker's top‑k NDCG. The authors find that various optimisation strategies—including prompt engineering, model scaling, sequential models, supervised neural rerankers, LoRA fine‑tuning, hybrid retrieval, score‑aware prompting, and LLM+CF fusion—do not significantly improve over a collaborative‑filtering baseline under realistic retrieval, and they propose a Recall‑Aware Evaluation Protocol (RAEP) to better assess rerankers in low‑recall regimes.

By Zhaohui Wang
arXiv Machine Learning
Jun 30

Diagnosing and Mitigating Retrieval Bottlenecks in LLM-Based Cold-Start Recommendation

arXiv:2606. 29947v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as rerankers in recommender systems, with the expectation that semantic understanding will help in cold-start and long-tail regimes.

By Zhe Dong (University of Maine at Presque Isle), Fang Qin (Stanford University), Manish Shah (Independent Researcher), Yicheng Wang (Independent Researcher)
arXiv Computation and Language
Nov 2

DecisionRank: On-Policy Distillation for Decision-Model Rerankers

DecisionRank introduces an on-policy distillation (OPD) method that trains decision models to replicate a generative LLM’s ranking knowledge by converting the student’s document scores into a ranked sequence and using a teacher to provide conditional feedback. The approach combines item‑space OPD with offline ranking supervision, achieving state‑of‑the‑art results on retrieval benchmarks such as TREC DL, BEIR, and BRIGHT. Even with a 1.7B parameter model, DecisionRank outperforms its 27B generative teacher on TREC DL19–23, and at 8B it surpasses the generative listwise counterpart while running 24.5× faster on TREC DL and 16.7× faster on BRIGHT.

By Yanran Wu, Dian Yu, Chongyang Bai, Jieyi Zhang, Rui Liu, Tianyi Chen, Shike Mei, Hang Cui, Frank Shyu, Sandeep Pandey, Luke Simon, Chunwei Liu, Xi Liu
arXiv Machine Learning
Sep 23

TailSpec-EASE: Knowledge-Graph-Regularized Linear Recommendation for Web Long-Tail Discovery

TailSpec-EASE is a lightweight linear recommender that incorporates a relation‑aware spectral knowledge‑graph prior into a local closed‑form reconstruction objective. By adapting the prior strength to item popularity, it provides stronger semantic guidance for long‑tail items. Across four public benchmarks, it achieves a favorable balance of overall accuracy, long‑tail performance, and training cost, improving NDCG@20 by up to 24% over a no‑KG baseline and training in just 37 seconds on CPU compared to thousands of seconds for GPU‑based KGAT and CPU LightGCN.

By Jianru Shen