The paper introduces epistemic warrant, a decision-level construct that assesses the stability and scope of a large language model’s preference for individual recommendations. It operationalizes this concept with a four-tier reliance certificate—unstable, context-dependent, locally supported, and broadly supported—applied to pairwise recommendations. Validation shows that stronger warrants align with expert and crowd consensus, provide distinct information from verbalized confidence, and are not merely a reflection of decision difficulty.
The paper investigates how large language models (LLMs) can evaluate explanations in recommender systems. It generates 18 explanation prototypes and has 14 LLMs rate them, comparing the results to human ratings from a user study. Findings show that while LLMs mimic human rating patterns and correlate moderately with human judgments, their absolute agreement is low and varies with model size and evaluation design, leading to four practical recommendations for using LLMs in this context.
By Kathrin Wardatzky, Oana Inel, Luca Rossetto, Abraham Bernstein
RPCBench is a new benchmark designed to evaluate large language models’ ability to critique recommendation requests by detecting, diagnosing, and handling flawed premises. It includes evidence‑grounded test instances across five recommendation domains and ten types of premise failures, and introduces a fine‑grained evaluation framework covering detection, error localization, handling strategy, and evidence faithfulness. Experiments with 11 LLMs reveal that proactive detection is the main bottleneck, with models struggling most on underspecified‑premise errors and showing that optimal critique quality occurs at intermediate reasoning lengths.
By Zhongru Chen, Yuan Wu, Yi Chang
ReFIne is a training framework that augments large reasoning models with three trustworthiness properties: interpretability, faithfulness, and reliability. It combines supervised fine‑tuning with GRPO to produce structured, tag‑based reasoning traces, explicitly disclose decisive information, and provide self‑assessments of soundness and confidence. Applied to Qwen3 models, ReFIne improves interpretability by 44.0 %, faithfulness by 18.8 %, and reliability by 42.4 % on mathematical benchmarks.
By Chung-En Sun, Ge Yan, Akshay Kulkarni, Tsui-Wei Weng
arXiv:2607. 20529v1 Announce Type: cross Abstract: Large Language Model (LLM) ensembles are increasingly used to improve reliability by combining predictions from multiple LLMs.
By Jiawei Zheng, Jiazhen Zhang
The paper introduces PROVE-REC, a two‑pass framework that generates verifiable preference proofs for large language model (LLM) recommendation systems. In Pass A, the model condenses user interaction histories into a compact proof of positive and avoidance claims linked to specific evidence entries. Pass B then uses only this proof and its evidence to predict the next item, ensuring the recommendation follows the reasoning path. Verification steps compare masked evidence and removed claims to confirm grounding and influence, while a ranking‑preservation objective retains useful historical information. Experiments on diverse real‑world datasets show PROVE‑REC outperforms strong baselines by up to 7.45%, producing claims that are both better grounded and more influential to recommendation quality.
By Yu Hou, Nathaniel Kang, Pengkai Wang, Hua Li