arXiv AI June 29, 2026 By Arnav Raj

PEBS: Per-rater Empirical-Bayes Shrinkage for RLHF Reward-Model Calibration

arXiv:2606. 27578v1 Announce Type: cross Abstract: Reward models for Reinforcement Learning from Human Feedback (RLHF) pool preferences across thousands of annotators and fit one global affine calibrator, collapsing raters with systematically different rating-scale offsets and slopes into a single average-rater fit that does not match any individual annotator.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

reinforcement-learning

arXiv AI

Jul 21

Rater State Bias in RLHF Preference Data: An Audit Framework

arXiv:2607. 16195v1 Announce Type: new Abstract: We identify a structured confound in Reinforcement Learning from Human Feedback (RLHF).

By Elena Kopteva, Vitaliy Hlynianyi-Zhuk

reinforcement-learning safety

arXiv Machine Learning

Jun 18

Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewards

arXiv:2606. 18963v1 Announce Type: new Abstract: We study online reward-punishment learning when the environment provides no scalar reward or evaluative label.

By Zirong Li

agents

arXiv AI

Jun 3

When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming

arXiv:2606. 03238v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) makes large-scale post-training possible by replacing an underspecified human objective with learned and scalable proxies.

By Zelalem Abahana

llms reinforcement-learning safety

arXiv AI

Jul 21

OR Else: A Differentiable Trust Region for Policy Optimization

arXiv:2607. 18163v1 Announce Type: cross Abstract: PPO and the GRPO baseline studied here use clipped surrogate objectives whose favorable-direction saturation introduces an abrupt change in the scalar objective's derivative.

By Chinmay Rane, Kanishka Tyagi, Michael Manry

llms reinforcement-learning

arXiv AI

6d ago

Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL

arXiv:2608. 11669v1 Announce Type: cross Abstract: Reinforcement learning against rubrics, lists of criteria graded by an LLM judge, has become a standard way to post-train language models on tasks with no deterministic answer.

By Minglai Yang, Xinyu Guo, Utkarsh Tyagi, Mian Zhang, Razvan Dumitru, Sunjie Hou, Yunzhong He, Daniel Yue Zhang, Ying Liu

llms reinforcement-learning benchmarks safety

arXiv AI

2d ago

Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation

arXiv:2608. 13564v1 Announce Type: new Abstract: Evaluating language-model agents at scale increasingly relies on a second language model as an automatic judge, because the gold signal, an executable environment reward, is expensive, slow, or unavailable at deployment time.

By Darragh Quinn, David Dylan, Roisin Healy, Fionn Carroll, Maeve Donnelly, Cormac Sheehan

llms agents fine-tuning

Related stories

Rater State Bias in RLHF Preference Data: An Audit Framework

Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewards

When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming

OR Else: A Differentiable Trust Region for Policy Optimization

Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL

Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation