arXiv Machine Learning July 10, 2026 By Yikai Wang, Shang Liu, Jose Blanchet

Wasserstein Distributionally Robust Regret Optimization for Reinforcement Learning from Human Feedback

Read the original on arXiv Machine Learning →

arXiv:2605. 00155v3 Announce Type: replace Abstract: Reinforcement learning from human feedback (RLHF) is a central post-training tool for aligning large language models, but its training reward is only a learned proxy for true human utility.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 30

Wasserstein Distributionally Robust Regret Optimization

arXiv:2504. 10796v4 Announce Type: replace-cross Abstract: Distributionally robust optimization (DRO) is widely used for decision-making under uncertainty, but its adversarial focus on worst-case loss can lead to overly conservative policies.

By Lukas-Benedikt Fiechtner, Jose Blanchet

safety

arXiv AI

Jun 30

Distributionally Robust Reinforcement Learning with Human Feedback

arXiv:2503. 00539v2 Announce Type: replace-cross Abstract: Reinforcement learning from human feedback (RLHF) has evolved to be one of the main methods for fine-tuning large language models (LLMs).

By Debmalya Mandal, Paulius Sasnauskas, Goran Radanovic

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 18

Wasserstein Policy Learning for Distributional Outcomes

arXiv:2606. 19117v1 Announce Type: cross Abstract: Offline policy learning has received growing attention in causal inference.

By Yiyan Huang, Cheuk Hang Leung, Qi Wu, Zhiheng Zhang

arXiv AI

Jun 9

A Unifying Lens on Reward Uncertainty in RLHF

arXiv:2606. 09073v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) is bottlenecked by \emph{reward hacking}, where the policy exploits errors in a proxy reward model (RM) and produces high RM scores without genuine quality gains.

By Ely Hahami, Yoel Zimmermann, Ray Zhou, Jack Benarroch Jedlicki

reinforcement-learning

arXiv AI

Jun 2

Efficient Exploration for Iterative Nash Preference Optimization

arXiv:2606. 01382v1 Announce Type: cross Abstract: Preference alignment is central to improving large language models, but standard reward-based formulations can be restrictive when human preferences are cyclic, non-transitive, or otherwise not representable by a scalar reward.

By Tianlong Nan, Xiaopeng Li, Christian Kroer, Tianyi Lin

llms fine-tuning benchmarks safety

arXiv AI

Jun 9

A Regret Minimization Framework on Preference Learning in Large Language Models

arXiv:2606. 09124v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has enabled progress on reasoning-intensive tasks by relying on task-specific verifiers that provide automated correctness signals.

By Suhwan Kim, Taehyun Cho, Geon-Hyeong Kim, Yu Jin Kim, Youngsoo Jang, Moontae Lee, Jungwoo Lee

llms reinforcement-learning benchmarks

Related stories

Wasserstein Distributionally Robust Regret Optimization

Distributionally Robust Reinforcement Learning with Human Feedback

Wasserstein Policy Learning for Distributional Outcomes

A Unifying Lens on Reward Uncertainty in RLHF

Efficient Exploration for Iterative Nash Preference Optimization

A Regret Minimization Framework on Preference Learning in Large Language Models