The paper investigates how large language models (LLMs) can evaluate explanations in recommender systems. It generates 18 explanation prototypes and has 14 LLMs rate them, comparing the results to human ratings from a user study. Findings show that while LLMs mimic human rating patterns and correlate moderately with human judgments, their absolute agreement is low and varies with model size and evaluation design, leading to four practical recommendations for using LLMs in this context.
By Kathrin Wardatzky, Oana Inel, Luca Rossetto, Abraham Bernstein
arXiv:2608. 10447v1 Announce Type: cross Abstract: Large language model-based recommender systems are increasingly adopting slow-thinking models that generate step-by-step reasoning before making predictions, often achieving higher accuracy than fast-thinking models that predict directly.
By Linh Dieu Le, Tong Chen, Shazia Sadiq, Hongzhi Yin, Ming Jin, Junliang Yu
The paper introduces a lifecycle framework for LLM-as-a-Judge systems used to evaluate recommendation explanations at Netflix. It outlines four phases—Birth, Training, Deployment, and Monitoring—detailing how each stage addresses specific technical and operational challenges. The authors report that after five weeks of A/B testing, judge-aligned explanations increased novel content viewing and successful browse-to-play sessions without quality takedowns.
By Emma Yanyang Kong, JJ Tan, Ishan Gupta, Lars Olds, Claire Campbell, David Fagnan, Veli Balin, Rohan Gosain, Louis Garcia, Minsu Jang
arXiv:2308.12241v2 Announce Type: replace-cross
Abstract: Recently, the fast development of Large Language Models (LLMs) such as ChatGPT has significantly advanced NLP tasks by enhancing the capabili...
By Junling Liu, Chao Liu, Peilin Zhou, Qichen Ye, Dading Chong, Kang Zhou, Yueqi Xie, Yuwei Cao, Shoujin Wang, Chenyu You, Philip S. Yu
The paper introduces PROVE-REC, a two‑pass framework that generates verifiable preference proofs for large language model (LLM) recommendation systems. In Pass A, the model condenses user interaction histories into a compact proof of positive and avoidance claims linked to specific evidence entries. Pass B then uses only this proof and its evidence to predict the next item, ensuring the recommendation follows the reasoning path. Verification steps compare masked evidence and removed claims to confirm grounding and influence, while a ranking‑preservation objective retains useful historical information. Experiments on diverse real‑world datasets show PROVE‑REC outperforms strong baselines by up to 7.45%, producing claims that are both better grounded and more influential to recommendation quality.
By Yu Hou, Nathaniel Kang, Pengkai Wang, Hua Li
Large language model-based recommender systems are increasingly adopting slow-thinking models that generate step-by-step reasoning before making predictions, often achieving higher accuracy than fast-thinking models that predict directly. However, their reasoning traces are often unnecessarily verbose, increasing inference costs without commensurate accuracy gains.
arXiv:2607. 23647v1 Announce Type: cross Abstract: Large language models (LLMs) can summarize heterogeneous user evidence in natural language, but current LLM recommenders often collapse enduring preferences, transient intent, and exposure-induced behavior into one profile.
By Gengyu Zhan
arXiv:2606. 10156v1 Announce Type: cross Abstract: As recommender systems transition toward agentic, multi-turn conversational interfaces, evaluation paradigms have struggled to keep pace.
By Bharath Sivaram Narasimhan, Karthik R Narasimhan
The paper presents a method for fine‑tuning a large language model (LLM) recommender to generate personalized, non‑harmful explanations for its recommendations. By training two LLM‑judge reward models and using constrained GRPO, the authors achieve a significant increase in the PASS rate for all three criteria, from 0.649 to 0.956 on their own judges and from 0.677 to 0.931 on an independent judge. The fine‑tuned model maintains its original recommendation performance, demonstrating that LLM‑based recommenders can be adapted to complex tasks without loss of effectiveness.
By Jiashu He, Emma Yanyang Kong, JJ Tan, David Fagnan
arXiv:2609.37216v1 Announce Type: cross
Abstract: Large language models can generate plausible code-review comments, but such comments may contain technically incorrect claims that mislead developers...
By Yue Pan, Jiawei Li, Ziyuan Zhang, Xiangxin Zhao, He Ye
arXiv:2606. 14142v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly adopted as backbones for Generative Recommendation (GR), promising access to pretrained world knowledge.
By Yinhan He, Liam Collins, Bhuvesh Kumar, Jundong Li, Neil Shah, Donald Loveland
arXiv:2608. 08889v1 Announce Type: new Abstract: Recommendation systems thrive on personalization, where ''correctness'' is rarely a binary truth but a matter of subjective human preference.
By Juncheng Dong, Ding Tong, Ishan Gupta, Yuyan Wang