arXiv AI

PAFO: Pareto Fairness Optimization for Personalized Reward Modeling

arXiv:2606. 07988v1 Announce Type: new Abstract: Large language models (LLMs) increasingly rely on reward models to align their outputs with diverse user preferences.

arXiv Machine Learning

Aug 4

Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning

arXiv:2608. 01556v1 Announce Type: new Abstract: Large language models are increasingly aligned to human preferences via reward modeling, but user preference data are sensitive and often cannot be centralized.

By Seongyoon Kim, Boryeong Cho, Jihwan Oh, Seokhyun Chung, Se-Young Yun

llms reinforcement-learning fine-tuning

arXiv AI

Jun 18

Steerable Cultural Preference Optimization of Reward Models

arXiv:2606. 18606v1 Announce Type: cross Abstract: It is essential for large language model (LLM) technology to serve many different cultural sub-communities in a manner that is acceptable to each community.

By Minsik Oh, Advit Deepak, Sophie Wu, Douwe Kiela, Ekaterina Shutova

llms reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 31

Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization

arXiv:2604. 07343v2 Announce Type: replace-cross Abstract: Pluralistic alignment has emerged as a critical frontier in the development of Large Language Models (LLMs), with reward models (RMs) serving as a central mechanism for capturing diverse human values.

By Qiyao Ma, Dechen Gao, Rui Cai, Boqi Zhao, Hanchu Zhou, Junshan Zhang, Zhe Zhao

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 16

Multi-User Dueling Bandits: A Fair Approach using Nash Social Welfare

arXiv:2605. 01961v2 Announce Type: replace Abstract: Learning from human preference data is becoming a useful tool, from fine-tuning large language models to training reinforcement learning agents.

By Maheed H. Ahmed, Mahsa Ghasemi

llms agents reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jun 9

Optimal Fair Aggregation of Crowdsourced Noisy Labels using Demographic Parity Constraints

arXiv:2601. 23221v2 Announce Type: replace Abstract: As acquiring reliable ground-truth labels is usually costly, or infeasible, crowdsourcing and aggregation of noisy human annotations is the typical resort.

By Gabriel Singer, Samuel Gruffaz, Olivier Vo Van, Nicolas Vayatis, Argyris Kalogeratos

benchmarks safety

arXiv AI

Aug 12

Procedural Fairness Failures in RLHF from Preference Averaging

arXiv:2608. 10126v1 Announce Type: cross Abstract: Reinforcement Learning from Human Feedback (RLHF) aggregates heterogeneous preferences into a single reward model, assuming preference homogeneity.

By M P V S Gopinadh, Karthik Kamuju, Kummari Avinash, John Joshua, Srinivasa Raju Rudraraju

llms agents reinforcement-learning safety

arXiv AI

Jun 4

Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modeling

arXiv:2606. 04284v1 Announce Type: cross Abstract: Preference modeling plays a central role in reinforcement learning from human feedback (RLHF), enabling large language models (LLMs) to align with human values.

By Yifan Wang, Jinyi Mu, Mayank Jobanputra, Yu Wang, Ji-Ung Lee, Soyoung Oh, Isabel Valera, Vera Demberg

llms reinforcement-learning safety

arXiv Machine Learning

Aug 10

Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs

arXiv:2509. 16462v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and economic disparities.

By Mina Arzaghi, Alireza Dehghanpour Farashah, Florian Carichon, Jean-Fran\c{c}ois Plante, Golnoosh Farnadi

llms rag fine-tuning benchmarks safety

arXiv AI

Jun 2

Demystifying the Optimal Fair Classifier in Multi-Class Classification

arXiv:2606. 00656v1 Announce Type: cross Abstract: Ensuring fair and equitable treatment across diverse groups, particularly in multi-class classification tasks, poses a significant challenge due to the persistent biases inherent in machine learning models.

By Li Zhang, Yuyuan Li, XiaoHua Feng, Jiaming Zhang, Fengyuan Yu, Chaochao Chen

fine-tuning safety

arXiv Machine Learning

Jun 26

Scoring Is Not Enough: Addressing Gaps in Utility-fairness Trade-offs for Ranking

arXiv:2606. 26369v1 Announce Type: cross Abstract: Scoring functions are used to represent the relevance of individual documents.

By Shubham Singh, Ian A. Kash, Mesrob I. Ohannessian

safety

arXiv Machine Learning

Aug 4

Inference-Time Policy Alignment for Fair Reinforcement Learning

arXiv:2608. 00175v1 Announce Type: new Abstract: Deep reinforcement learning (RL) agents achieve strong performance by optimizing scalar reward functions.

By Umer Siddique, Peilang Li, Conor Wallace, Yongcan Cao

llms agents reinforcement-learning safety

arXiv AI

5d ago

FSGR: Mitigating Token Frequency Bias for Fair SID-Based Generative Recommendation

arXiv:2608. 12845v1 Announce Type: cross Abstract: Semantic ID (SID)-based generative recommendation has recently achieved remarkable success.

By Yuchen Zheng, Sihan Xu, Jingwen Yang, Xiangrui Cai, Haiwei Zhang, Xiaojie Yuan

llms fine-tuning safety

Related stories