arXiv Machine Learning August 27, 2026 By Hyunji Nam, Yanming Wan, Mickel Liu, Peter Ahnn, Jianxun Lian, Natasha Jaques

Learning to summarize user information for personalized reinforcement learning from human feedback

Read the original on arXiv Machine Learning →

The paper introduces PLUS, a framework that uses reinforcement learning to generate text-based summaries of individual users’ preferences, characteristics, and past conversations. These summaries condition a reward model, allowing it to predict personalized response preferences and improving reward accuracy by 11–77 % over the standard Bradley‑Terry model. PLUS demonstrates robust performance with new users and topics, achieves a 25 % improvement over existing personalized RLHF techniques, and enables zero‑shot personalization for state‑of‑the‑art models like GPT‑4.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 11

Learning Preference Adaptation for Large Language Model Personalization via Verbal Reinforcement Learning

arXiv:2608. 09507v1 Announce Type: cross Abstract: Natural language user preferences provide an interpretable interface for LLM personalization.

By Yuting Liu, Wei Wu, Jianzhe Zhao, Guibing Guo

llms rag agents reinforcement-learning

arXiv AI

Jun 8

Re-Centering Humans in LLM Personalization

arXiv:2606. 06614v1 Announce Type: cross Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data.

By Lechen Zhang, Jiarui Liu, Tal August

llms

arXiv AI

Jun 11

GrowLoop: Self-Evolving Conversation Evaluation Seeded by Human

arXiv:2605. 28882v2 Announce Type: replace-cross Abstract: With the rapid advancement of large language models, evaluating human-likeness in open-ended conversation has become increasingly important.

By Yihang Lin, Yunze Gao, Zeyang Lin, Dongbo Li, Kun Peng, Yue Liu

llms agents benchmarks safety

arXiv AI

Jun 2

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization

arXiv:2510. 05342v2 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a simple and effective method for aligning large language models.

By Hyung Gyu Rho

llms nlp reinforcement-learning

arXiv AI

Jul 22

S2T-RLHF: Hierarchical Credit Assignment for Stable Preference-Based RLHF

arXiv:2607. 18258v1 Announce Type: new Abstract: Reinforcement learning from human feedback (RLHF) with preference-based reward models often exhibits unstable training dynamics.

By Wei Chen, Guanghui Zhu, Yafei Li, Limin Wang, Yihua Huang

reinforcement-learning safety

arXiv Machine Learning

Jun 30

Synthetic Interaction Data for Scalable Personalization in Large Language Models

arXiv:2602. 12394v2 Announce Type: replace Abstract: Personalized prompting offers large opportunities for deploying large language models (LLMs) to diverse users, yet existing prompt optimization methods primarily focus on task-level optimization while largely overlooking user-specific preferences and latent constraints of individual users.

By Yuchen Ma, Yue Huang, Wenjie Wang, Xiaonan Luo, Xiangliang Zhang, Stefan Feuerriegel

llms agents reinforcement-learning benchmarks safety

One email a morning, machine-written

Related stories

Learning Preference Adaptation for Large Language Model Personalization via Verbal Reinforcement Learning

Re-Centering Humans in LLM Personalization

GrowLoop: Self-Evolving Conversation Evaluation Seeded by Human

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization

S2T-RLHF: Hierarchical Credit Assignment for Stable Preference-Based RLHF

Synthetic Interaction Data for Scalable Personalization in Large Language Models