The paper introduces the concept of epistemic warrant to assess when users should rely on individual recommendations from large language models (LLMs). It defines a four-tier reliance certificate—unstable, context-dependent, locally supported, and broadly supported—based on the stability and scope of a model’s preference. Experiments show that these warrants align with expert judgments and crowd consensus, and they provide distinct information from confidence scores or decision difficulty.
By Shai Vardi, Jo\~ao Sedoc
The paper introduces epistemic warrant, a decision-level construct that assesses the stability and scope of a large language model’s preference for individual recommendations. It operationalizes this concept with a four-tier reliance certificate—unstable, context-dependent, locally supported, and broadly supported—applied to pairwise recommendations. Validation shows that stronger warrants align with expert and crowd consensus, provide distinct information from verbalized confidence, and are not merely a reflection of decision difficulty.
The paper introduces PROVE-REC, a two‑pass framework that generates verifiable preference proofs for large language model (LLM) recommendation systems. In Pass A, the model condenses user interaction histories into a compact proof of positive and avoidance claims linked to specific evidence entries. Pass B then uses only this proof and its evidence to predict the next item, ensuring the recommendation follows the reasoning path. Verification steps compare masked evidence and removed claims to confirm grounding and influence, while a ranking‑preservation objective retains useful historical information. Experiments on diverse real‑world datasets show PROVE‑REC outperforms strong baselines by up to 7.45%, producing claims that are both better grounded and more influential to recommendation quality.
By Yu Hou, Nathaniel Kang, Pengkai Wang, Hua Li
CARTBench is a new benchmark for evaluating large language model shopping assistants on complex, ambiguous requests that involve timelines. It compiles 1,603 timestamped dialogues from 17 retail domains and pairs them with product records, then uses case‑specific rubrics based on Ideal Response Principles to score responses across Core, Excellence, and Safety criteria. In a full‑suite evaluation of ten commercial APIs and several open‑weight models, only three models scored above 90% on Core and none surpassed 45% on Excellence, revealing a gap between required behavior and high‑quality purchase advice.
By Yuqin Dai, Zhanwei Zhang, Honghui Yang, Zhouheng Yao, Yueqi Zhang, Jingyu He, Chaoyang Shi, Fanding Wei, Boyuan Pan, Kai Xu, Yao Hu, Yongliang Shen
RPCBench is a new benchmark designed to evaluate large language models’ ability to critique recommendation requests by detecting, diagnosing, and handling flawed premises. It includes evidence‑grounded test instances across five recommendation domains and ten types of premise failures, and introduces a fine‑grained evaluation framework covering detection, error localization, handling strategy, and evidence faithfulness. Experiments with 11 LLMs reveal that proactive detection is the main bottleneck, with models struggling most on underspecified‑premise errors and showing that optimal critique quality occurs at intermediate reasoning lengths.
By Zhongru Chen, Yuan Wu, Yi Chang
arXiv:2608.21243v1 Announce Type: cross
Abstract: Sequential recommendation predicts the next item from a user's interaction history, but not every interaction is equally informative. Real logs combi...
By Zichun Jin, Zihan Zhou, Yinan Liu, Bin Wang, Xiaochun Yang