arXiv Machine Learning July 21, 2026 By Qining Zhang, Lei Ying

Sign-SZPO: Provable Preference-based Reinforcement Learning with an Unknown Link Function

Read the original on arXiv Machine Learning →

arXiv:2506. 03066v2 Announce Type: replace Abstract: The link function, which characterizes the relationship between the preference for two trajectories and their returns, is a crucial component in designing RL algorithms that learn from preference feedback.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

reinforcement-learning

arXiv AI

Aug 5

SP3O: Reinforcement Learning from Segment Preferences without Reward Modeling

arXiv:2608. 02951v1 Announce Type: cross Abstract: Preference-based reinforcement learning (PbRL) for general stochastic MDPs often requires training a reward model.

By Evan Assmus, Qining Zhang, Lei Ying

llms reinforcement-learning robotics fine-tuning

arXiv AI

Jun 18

UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning

arXiv:2606. 19328v1 Announce Type: cross Abstract: Preference-based RL provides an approach to learning reward models from pairwise comparisons of behaviors, bypassing the need for explicit reward design.

By Mohamed Nabail, Leo Cheng, Jingmin Wang, Nicholas Rhinehart

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 11

PAWS: Preference Learning with Advantage-Weighted Segments

arXiv:2606. 11982v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) learns policies from human trajectory-level comparisons, avoiding explicit reward design and expert demonstrations.

By Aleksandar Taranovic, Onur Celik, Niklas Freymuth, Ge Li, Serge Thilges, Huy Le, Tai Hoang, Rania Rayyes, Gerhard Neumann

reinforcement-learning robotics

arXiv AI

Jun 4

Semiparametric Preference Optimization: Your Language Model is Secretly a Single-Index Model

arXiv:2512. 21917v3 Announce Type: replace-cross Abstract: Policy alignment to preference data typically assumes a known link function between observed preferences and latent rewards (e.

By Nathan Kallus

llms safety

arXiv AI

Jun 2

Efficient Exploration for Iterative Nash Preference Optimization

arXiv:2606. 01382v1 Announce Type: cross Abstract: Preference alignment is central to improving large language models, but standard reward-based formulations can be restrictive when human preferences are cyclic, non-transitive, or otherwise not representable by a scalar reward.

By Tianlong Nan, Xiaopeng Li, Christian Kroer, Tianyi Lin

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jul 20

Robust Peak-cost Constrained Reinforcement Learning

arXiv:2607. 15457v1 Announce Type: new Abstract: We study robust peak-cost constrained reinforcement learning (RP-CRL), where the objective is to maximize expected reward while controlling the maximum cost encountered along a trajectory.

By Shilpa Mukhopadhyay, Sourav Ganguly, Santosh Mohan Rajkumar, Honghao Wei, Debdipta Goswami, Arnob Ghosh

reinforcement-learning safety

Related stories

SP3O: Reinforcement Learning from Segment Preferences without Reward Modeling

UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning

PAWS: Preference Learning with Advantage-Weighted Segments

Semiparametric Preference Optimization: Your Language Model is Secretly a Single-Index Model

Efficient Exploration for Iterative Nash Preference Optimization

Robust Peak-cost Constrained Reinforcement Learning