arXiv Machine Learning August 4, 2026 By Seongyoon Kim, Boryeong Cho, Jihwan Oh, Seokhyun Chung, Se-Young Yun

Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning

Read the original on arXiv Machine Learning →

arXiv:2608. 01556v1 Announce Type: new Abstract: Large language models are increasingly aligned to human preferences via reward modeling, but user preference data are sensitive and often cannot be centralized.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

1d ago

Global Federated Learning Strategies for Building Efficient Personalized Models

arXiv:2608. 15107v1 Announce Type: new Abstract: Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously.

By Seongyoon Kim

reinforcement-learning fine-tuning efficiency safety

arXiv AI

Jun 9

PAFO: Pareto Fairness Optimization for Personalized Reward Modeling

arXiv:2606. 07988v1 Announce Type: new Abstract: Large language models (LLMs) increasingly rely on reward models to align their outputs with diverse user preferences.

By Xiaoyan Zhao, Haoting Ni, Yang Zhang, Chunyuan Zheng, Haoxuan Li, Fuli Feng

llms reinforcement-learning safety

arXiv Machine Learning

Jun 30

Personalized Additive Modeling for Multi-level Federated Learning

arXiv:2405. 16472v2 Announce Type: replace Abstract: Contemporary AI faces the challenge of balancing generality with user-specific personalization.

By Shutong Chen, Guodong Long, Tianyi Zhou, Jie Ma, Jing Jiang, Chengqi Zhang

arXiv Machine Learning

Jun 16

GD$^2$PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization

arXiv:2606. 16771v1 Announce Type: new Abstract: As LLMs advance, post-training reinforcement learning (RL) increasingly relies on multi-dimensional rewards to cultivate comprehensive capabilities.

By Haotian Liu, Yihao Liu, Jingwei Ni, Siyuan Huang, Xinpeng Liu, Pengyu Cheng, Jiajun Song, Ruijin Ding, Junfeng Li, Zhechao Yu, Mengyu Zhou, Hongteng Xu, Xiaoxi Jiang, Guanjun Jiang

llms agents reinforcement-learning safety

arXiv AI

Jun 18

Steerable Cultural Preference Optimization of Reward Models

arXiv:2606. 18606v1 Announce Type: cross Abstract: It is essential for large language model (LLM) technology to serve many different cultural sub-communities in a manner that is acceptable to each community.

By Minsik Oh, Advit Deepak, Sophie Wu, Douwe Kiela, Ekaterina Shutova

llms reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 30

Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback

arXiv:2607. 26094v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback (RLHF) is the standard approach for aligning large language models with human preferences, but its quality is limited by static, task-agnostic reward models.

By Yunpeng Chu

llms reinforcement-learning benchmarks safety

Related stories

Global Federated Learning Strategies for Building Efficient Personalized Models

PAFO: Pareto Fairness Optimization for Personalized Reward Modeling

Personalized Additive Modeling for Multi-level Federated Learning

GD$^2$PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization

Steerable Cultural Preference Optimization of Reward Models

Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback