arXiv Machine Learning
IncentRL: The Trade-Off Between Preference Guidance and Task Performance
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 1
Locally-Guided Actor-Critic: Training a Goal-conditioned Actor with a Subgoal-aware Critic
arXiv:2608.30406v1 Announce Type: new Abstract: Goal-conditioned reinforcement learning struggles with long horizons when rewards are sparse. While a planner can provide subgoals to guide a low-level...
arXiv AI
Jun 12
Boosting Direct Preference Optimization with Penalization
arXiv:2606. 12505v1 Announce Type: cross Abstract: Offline preference optimization has become a practical substitute for reinforcement learning from human feedback, but pairwise objectives such as Direct Preference Optimization (DPO) and its variants use only the chosen and rejected responses stored in a static dataset.
arXiv Computation and Language
Aug 25
Length-Controlled Margin-Based Preference Optimization without Reference Model
arXiv:2502.14643v3 Announce Type: replace Abstract: Direct Preference Optimization (DPO) is a widely adopted offline algorithm for preference-based reinforcement learning from human feedback (RLHF),...
arXiv AI
Aug 5
SP3O: Reinforcement Learning from Segment Preferences without Reward Modeling
arXiv:2608. 02951v1 Announce Type: cross Abstract: Preference-based reinforcement learning (PbRL) for general stochastic MDPs often requires training a reward model.
arXiv AI
Jun 26
Automating Potential-based Reward Shaping with Vision Language Model Guidance
arXiv:2606. 27180v1 Announce Type: cross Abstract: Sparse rewards are inherently challenging for reinforcement learning agents as they lack intermediate feedback to guide exploration and to correctly attribute the sparse success rewards to relevant parts of the trajectory.
arXiv AI
Jun 2
Efficient Exploration for Iterative Nash Preference Optimization
arXiv:2606. 01382v1 Announce Type: cross Abstract: Preference alignment is central to improving large language models, but standard reward-based formulations can be restrictive when human preferences are cyclic, non-transitive, or otherwise not representable by a scalar reward.