arXiv Machine Learning

Per-Query Gating of LLM Rerankers for Multi-Hop Retrieval

arXiv:2609. 22880v1 Announce Type: cross Abstract: LLM rerankers add of the order of \$0.

arXiv Machine Learning
Sep 21

Predictable Failure in Multi-Hop Retrieval: Score-Distributional Confidence Scoring and Abstention

The paper shows that multi‑hop retrieval failures cluster in predictable subpopulations and formalizes this with two theoretical results: (1) confident‑failure reduction is possible only when retrieval features carry mutual information about success, and (2) no single ANN score feature dominates across all failure regimes. Building on these insights, the authors introduce RegimeAbstain, which computes a Retrieval Confidence Score (RCS) from up to nine query‑ANN structural features and uses it to calibrate an abstention policy. Across three benchmarks and two retrieval architectures, RCS achieves the best or co‑best AUC‑AC and significantly reduces the Confident‑Wrong‑Answer Rate, demonstrating its effectiveness and domain‑agnostic applicability.

By Andre Bacellar
arXiv AI
Aug 20

Pairwise Ranking Outperforms Single-Action RL for Offline Explanation Selection: A Practical Lesson

The paper presents a cost‑effective approach for industrial explainable‑recommendation systems by decoupling explanation generation from selection. Candidate explanations are pre‑generated using six prompt styles and two commodity LLMs, then a lightweight CPU‑resident selector (e.g., LambdaRank) chooses the best one at request time, achieving sub‑100 ms latency without GPUs. Experiments on a 2,958‑pair Google Local subset and a 300‑pair MovieLens‑1M split show that pairwise ranking methods outperform single‑action RL baselines, while KG‑path selectors achieve near‑perfect user satisfaction scores.

By Tanay Chowdhury, Saeideh Shahrokh Esfahani
Hugging Face Trending Papers
Aug 19

Pairwise Ranking Outperforms Single-Action RL for Offline Explanation Selection: A Practical Lesson

The paper presents a cost‑effective approach for industrial explainable‑recommendation systems by decoupling explanation generation from selection. Explanations are pre‑generated using six prompt styles and two commodity LLMs, then a lightweight CPU‑resident selector (e.g., LambdaRank) chooses the best one at request time, achieving sub‑100 ms latency without GPUs. Experiments on a 2,958‑pair Google Local subset and a 300‑pair MovieLens‑1M split show that pairwise ranking outperforms single‑action RL methods, while KG‑path selectors achieve near‑perfect unique‑output rates, and the overall end‑to‑end build cost is around $15 on commodity hardware.

arXiv Machine Learning
Aug 13

LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence

arXiv:2608. 11922v1 Announce Type: cross Abstract: Predictive-distribution entropy makes a strong selection rule in retrieval-augmented question answering: across five QA benchmarks, keeping the candidate answer that a frozen respondent LLM produces with the lowest answer-token entropy lifts mean answer $F_1$ from 0.

By Po-Jen Ko, Che-Cheng Wu, Hung-Chun Hsu, Li-Yang Chang, Chuan-Ju Wang
arXiv Machine Learning
Aug 28

Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation

The paper investigates the trade‑off of using a shared search‑and‑recommendation index that scores new items purely from features, thereby keeping the index open to unseen items. Experiments on public logs show that a feature‑based tower can match warm‑item performance (Recall@20 0.9595 vs 0.9510) and a lexical baseline, while a full‑catalog check is inconclusive. The study also quantifies the cost of this openness on recommendation quality across several baselines, revealing that exact full‑softmax training improves recall but is impractical at catalog scale.

By Theodore Rogers, Joe Standerfer, Dmitrii Timoshenko, Haoxue Li, Zuhaib Akhtar, Soyoung Yang