arXiv Machine Learning

Offline Constrained RLHF with Multiple Preference Oracles

arXiv AI

Jun 2

Efficient Exploration for Iterative Nash Preference Optimization

arXiv:2606. 01382v1 Announce Type: cross Abstract: Preference alignment is central to improving large language models, but standard reward-based formulations can be restrictive when human preferences are cyclic, non-transitive, or otherwise not representable by a scalar reward.

By Tianlong Nan, Xiaopeng Li, Christian Kroer, Tianyi Lin

llms fine-tuning benchmarks safety

arXiv Machine Learning

Aug 4

Learning-Based Stochastic Optimal Control with Infinite-Horizon Probabilistic Constraints

arXiv:2608. 01151v1 Announce Type: cross Abstract: In this paper, we consider stochastic optimal control problems with infinite-horizon joint chance constraints.

By Francesco Cordiano, Kanghui He, Bart De Schutter

reinforcement-learning

arXiv AI

Sep 10

Provable Pluralistic Alignment: Multi-Party RLHF under Offline Human Feedback

arXiv:2403.05006v2 Announce Type: replace-cross Abstract: Pluralistic alignment requires learning from feedback that reflects persistent and potentially conflicting stakeholder preferences while ulti...

By Huiying Zhong, Tianwei Gao, Zhiwei Steven Wu, Linjun Zhang, Weijie J. Su, Zhun Deng

reinforcement-learning safety

arXiv AI

Jun 17

Learning Fair Pareto-Optimal Policies in Multi-Objective Reinforcement Learning

arXiv:2606. 18111v1 Announce Type: cross Abstract: Fairness is an important aspect of decision-making in multi-objective reinforcement learning (MORL), where policies must ensure both optimality and equity across multiple, potentially conflicting objectives.

By Umer Siddique, Peilang Li, Yongcan Cao

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 8

Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models

arXiv:2505. 10892v2 Announce Type: replace Abstract: Post-training LLMs with RLHF and preference optimization methods (e.

By Akhil Agnihotri, Rahul Jain, Deepak Ramachandran, Zheng Wen

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Sep 22

Transductive Off-policy Proximal Policy Optimization

arXiv:2406.03894v2 Announce Type: replace Abstract: Proximal Policy Optimization (PPO) is a popular model-free reinforcement learning algorithm, esteemed for its simplicity and efficacy. However, due...

By Yaozhong Gan, Renye Yan, Xiaoyang Tan, Zhe Wu, Junliang Xing

reinforcement-learning

arXiv Machine Learning

Jul 10

Wasserstein Distributionally Robust Regret Optimization for Reinforcement Learning from Human Feedback

arXiv:2605. 00155v3 Announce Type: replace Abstract: Reinforcement learning from human feedback (RLHF) is a central post-training tool for aligning large language models, but its training reward is only a learned proxy for true human utility.

By Yikai Wang, Shang Liu, Jose Blanchet

llms reinforcement-learning

arXiv AI

Jul 17

Fully Offline Reinforcement Learning

arXiv:2505. 22442v3 Announce Type: replace-cross Abstract: Offline RL (ORL) promises safe and sample-efficient deployment but existing methods rely on undocumented online interactions for hyperparameter tuning and lack reliable fully offline estimates of initial online performance.

By Mattie Fellows, Clarisse Wibault, Uljad Berdica, Johannes Forkel, Maike Osborne, Jakob N. Foerster

reinforcement-learning

arXiv Machine Learning

Jun 2

From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning

arXiv:2606. 01123v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) avoids explicit reward engineering by learning from pairwise human preference feedback.

By Jun-Jie Yang, Chia-Heng Hsu, Kui-Yuan Chen, Ping-Chun Hsieh

reinforcement-learning fine-tuning

arXiv AI

Jun 18

UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning

arXiv:2606. 19328v1 Announce Type: cross Abstract: Preference-based RL provides an approach to learning reward models from pairwise comparisons of behaviors, bypassing the need for explicit reward design.

By Mohamed Nabail, Leo Cheng, Jingmin Wang, Nicholas Rhinehart

reinforcement-learning benchmarks

arXiv Machine Learning

Sep 15

When Greedy Sampling Explores: KL-Regularized Contextual Bandits without Eluder-Dimension Dependence

arXiv:2609.13564v1 Announce Type: new Abstract: We study KL-regularized contextual bandits under both reward and preference feedback. We show that greedy sampling can achieve logarithmic regret witho...

By Zichen Wang, Haoyang Hong, Huazheng Wang

arXiv Machine Learning

Sep 21

Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards

arXiv:2603.27884v2 Announce Type: replace Abstract: We study safe reinforcement learning in finite-horizon linear mixture constrained Markov decision processes (CMDPs) with adversarial rewards under...

By Kihyun Yu, Seoungbin Bae, Dabeen Lee

reinforcement-learning safety

Related stories