arXiv Machine Learning
Robust Risk-Sensitive Reinforcement Learning from Corrupted Human Feedback
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 29
Uncertainty-Aware Reward Discounting for Mitigating Reward Hacking
arXiv:2604. 26360v2 Announce Type: replace-cross Abstract: Reinforcement learning from human feedback (RLHF) systems face a compounding alignment challenge: not only are learned reward models uncertain about unseen state-action pairs, but the human preference annotations they are trained on are themselves inconsistent, context-dependent, and noisy.
arXiv Machine Learning
Jul 10
Wasserstein Distributionally Robust Regret Optimization for Reinforcement Learning from Human Feedback
arXiv:2605. 00155v3 Announce Type: replace Abstract: Reinforcement learning from human feedback (RLHF) is a central post-training tool for aligning large language models, but its training reward is only a learned proxy for true human utility.
arXiv Machine Learning
Jun 19
Matching Markets meet Cumulative Prospect Theory: Towards Optimal and Adversarially Robust Learning
arXiv:2606. 19883v1 Announce Type: new Abstract: We study a multi-agent multi-armed bandit problem in the competitive setup with two-sided matching markets under a human centric decision making model.
arXiv AI
Sep 1
BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learning
BCPPO is a new variant of Proximal Policy Optimization that uses Bachelier-inspired cost‑prediction networks to generate a smooth penalty based on disagreement among critics. The method keeps temporal‑difference learning unchanged, applies a saturation‑aware controller to manage cost penalties, and deploys only the policy network. Across extensive experiments, BCPPO outperforms comparators in achieving higher mean returns while maintaining lower or comparable CVaR in all tested tasks.
arXiv AI
Jun 18
UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning
arXiv:2606. 19328v1 Announce Type: cross Abstract: Preference-based RL provides an approach to learning reward models from pairwise comparisons of behaviors, bypassing the need for explicit reward design.
arXiv AI
Jun 19
Uncertainty-Aware Reward Modeling for Stable RLHF
arXiv:2606. 19818v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) aligns large language models by training reward models on preference data and optimizing policies to maximize predicted rewards.