Hugging Face Trending Papers
PerSPL: Personalized Semi-supervised Preference Learning
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Oct 22
PerSPL: Personalized Semi-supervised Preference Learning
PerSPL (Personalized Semi-supervised Preference Learning) is a backbone‑agnostic framework that stabilizes cross‑prompt score coordinates using pair‑mean score centering, enabling the conversion of unlabeled responses into user‑specific training signals. It addresses the scarcity of user‑specific preference comparisons by leveraging large pools of unlabeled responses, which are otherwise unused by existing personalized reward models. Across four benchmarks—PersonalLLM, HelpSteer2, Reddit TLDR, and Chatbot Arena—PerSPL achieves the highest mean accuracy for both seen and unseen users, demonstrating its effectiveness in summarization and open‑domain chatbot interaction settings.
arXiv Machine Learning
Aug 27
Learning to summarize user information for personalized reinforcement learning from human feedback
The paper introduces PLUS, a framework that uses reinforcement learning to generate text-based summaries of individual users’ preferences, characteristics, and past conversations. These summaries condition a reward model, allowing it to predict personalized response preferences and improving reward accuracy by 11–77 % over the standard Bradley‑Terry model. PLUS demonstrates robust performance with new users and topics, achieves a 25 % improvement over existing personalized RLHF techniques, and enables zero‑shot personalization for state‑of‑the‑art models like GPT‑4.
arXiv Computation and Language
Oct 23
Memory-Aware Reward Modeling: Learning Human Preferences for LLM Memory Use
The paper investigates whether using a language model’s memory to personalize responses aligns with user preferences. By collecting about 1,000 pairwise judgments from 87 participants on ChatGPT queries that included personal memories, the authors find that memory‑conditioned responses are not consistently preferred and that users sometimes strongly favor non‑memory responses. An automatic reward model trained on these judgments achieves 72.8% preference prediction accuracy and outperforms existing baselines, highlighting the need for evaluation and optimization objectives that better reflect user preferences for memory use.
arXiv AI
Jun 8
Re-Centering Humans in LLM Personalization
arXiv:2606. 06614v1 Announce Type: cross Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data.
arXiv AI
Jul 7
Adaptive Margin RLHF via Preference over Preferences
arXiv:2509. 22851v4 Announce Type: replace-cross Abstract: Margin-based optimization is fundamental to improving generalization and robustness in classification tasks.
arXiv Machine Learning
Jul 31
Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization
arXiv:2604. 07343v2 Announce Type: replace-cross Abstract: Pluralistic alignment has emerged as a critical frontier in the development of Large Language Models (LLMs), with reward models (RMs) serving as a central mechanism for capturing diverse human values.