Hugging Face Trending Papers June 3, 2026

BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization

Read the original on Hugging Face Trending Papers →

Mitigating social bias in Large Language Models (LLMs) presents a distinct alignment challenge: unlike verifiable tasks, bias lacks a single ground truth, creating a high-variance, subjective reward landscape. Previous preference-based fine-tuning methods have major trade-offs: Direct Preference Optimization (DPO) is limited by the lack of exploration inherent in offline training, while Proximal Policy Optimization (PPO) can lead to training instability due to potentially unreliable critic estimates.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv AI

Jun 4

BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization

arXiv:2606. 04807v1 Announce Type: new Abstract: Mitigating social bias in Large Language Models (LLMs) presents a distinct alignment challenge: unlike verifiable tasks, bias lacks a single ground truth, creating a high-variance, subjective reward landscape.

By Saket Reddy, Ke Yang, ChengXiang Zhai

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 7

Unbiased Alignment for Large Language Models with Noisy Preferences

arXiv:2607. 03248v1 Announce Type: cross Abstract: The alignment of large language models with human preferences is commonly achieved through Reinforcement Learning from Human Feedback or Direct Preference Optimization.

By Jialiang Wang, Xianming Liu, Xiong Zhou, Hui Liu, Haoliang Li

llms reinforcement-learning benchmarks safety

arXiv AI

Jul 7

RSPO: Reward-Swap Policy Optimization for Multi-Turn LLM Agents

arXiv:2607. 04713v1 Announce Type: cross Abstract: Reinforcement learning holds significant potential for training large language models (LLMs) to handle multi-turn interactive tasks.

By Qiang Liu, Taian Guo, Ruizhi Qiao, Xing Sun

llms agents reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Aug 4

Inference-Time Policy Alignment for Fair Reinforcement Learning

arXiv:2608. 00175v1 Announce Type: new Abstract: Deep reinforcement learning (RL) agents achieve strong performance by optimizing scalar reward functions.

By Umer Siddique, Peilang Li, Conor Wallace, Yongcan Cao

llms agents reinforcement-learning safety

arXiv AI

Jun 11

Toward Preference-aligned Large Language Models via Residual-based Model Steering

arXiv:2509. 23982v2 Announce Type: replace-cross Abstract: Preference alignment is a critical step in making Large Language Models (LLMs) useful and aligned with (human) preferences.

By Lucio La Cava, Andrea Tagarelli

llms reinforcement-learning benchmarks safety

arXiv AI

Jun 2

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization

arXiv:2510. 05342v2 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a simple and effective method for aligning large language models.

By Hyung Gyu Rho

llms nlp reinforcement-learning

Related stories

BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization

Unbiased Alignment for Large Language Models with Noisy Preferences

RSPO: Reward-Swap Policy Optimization for Multi-Turn LLM Agents

Inference-Time Policy Alignment for Fair Reinforcement Learning

Toward Preference-aligned Large Language Models via Residual-based Model Steering

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization