arXiv AI
Unbiased Reward Modeling from Implicit Feedback for LLM Alignment
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jul 31
Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization
arXiv:2604. 07343v2 Announce Type: replace-cross Abstract: Pluralistic alignment has emerged as a critical frontier in the development of Large Language Models (LLMs), with reward models (RMs) serving as a central mechanism for capturing diverse human values.
arXiv AI
Jul 17
Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning
arXiv:2506. 12529v2 Announce Type: replace-cross Abstract: Preference-based Reinforcement Learning (PbRL) entails a variety of approaches for aligning models with human intent to alleviate the burden of reward engineering.
arXiv AI
Jul 7
MARS: Margin and Semantic-Aware Data Augmentation for Reward Modeling
arXiv:2602. 17658v3 Announce Type: replace-cross Abstract: Reward modeling is central to alignment pipelines such as RLHF, RLAIF, and PPO-based policy optimization, yet its reliability is constrained by limited and heterogeneous human preference data that are expensive to collect at scale.
arXiv AI
Sep 1
Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
The paper introduces Personalized Group Relative Policy Optimization (P‑GRPO), a new alignment framework for large language models that separates advantage estimation from immediate batch statistics. By normalizing advantages using preference‑group‑specific reward histories instead of the concurrent generation group, P‑GRPO maintains contrastive signals for distinct user preferences. Experiments across various tasks show that P‑GRPO converges faster and yields higher rewards than standard GRPO, improving alignment with heterogeneous human preferences while preserving general capabilities.
arXiv AI
Jul 7
Unbiased Alignment for Large Language Models with Noisy Preferences
arXiv:2607. 03248v1 Announce Type: cross Abstract: The alignment of large language models with human preferences is commonly achieved through Reinforcement Learning from Human Feedback or Direct Preference Optimization.
arXiv AI
Aug 5
Enhancing VLM Reward Models Through Structure-Aware Fine-Tuning
arXiv:2608. 03875v1 Announce Type: cross Abstract: Designing effective reward functions remains a major bottleneck in Reinforcement Learning (RL).