OpenAI Blog
Scaling laws for reward model overoptimization
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
arXiv AI
Jun 9
Reward Shaping for (Inference-Time) Alignment: A Stackelberg Game Perspective
arXiv:2602. 02572v2 Announce Type: replace-cross Abstract: Existing alignment methods directly use the reward model learned from user preference data to optimize an LLM policy, subject to KL regularization with respect to the base policy.
arXiv Machine Learning
Jul 22
On Incentivized Exploration beyond Bayesianism and Full-Information
arXiv:2607. 18300v1 Announce Type: cross Abstract: We extend Incentive Compatible Exploration beyond the Bayesian full-information setting of Kremer et al.
arXiv Machine Learning
Jul 7
Explicit Credit Assignment through Local Rewards and Dependence Graphs in Multi-Agent Reinforcement Learning
arXiv:2601. 21523v2 Announce Type: replace Abstract: To promote cooperation in Multi-Agent Reinforcement Learning, the reward signals of all agents can be aggregated together, forming global rewards that are commonly known as the fully cooperative setting.
arXiv Machine Learning
Aug 11
Directional-Clamp PPO
arXiv:2511. 02577v2 Announce Type: replace Abstract: Proximal Policy Optimization (PPO) is widely regarded as one of the most successful deep reinforcement learning algorithms, known for its robustness and effectiveness across a range of problems.
arXiv AI
Aug 5
SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation
arXiv:2608. 03092v1 Announce Type: cross Abstract: We aim to improve model performance in multi-reward reinforcement learning training process.
arXiv AI
Aug 12
Procedural Fairness Failures in RLHF from Preference Averaging
arXiv:2608. 10126v1 Announce Type: cross Abstract: Reinforcement Learning from Human Feedback (RLHF) aggregates heterogeneous preferences into a single reward model, assuming preference homogeneity.