arXiv Machine Learning
Jun 5
Reward Learning through Ranking Mean Squared Error
arXiv:2601. 09236v3 Announce Type: replace Abstract: Reward design remains a significant bottleneck in applying reinforcement learning (RL) to real-world problems.
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction
arXiv:2608. 06310v1 Announce Type: new Abstract: Recent advances in reward modeling show a paradigm shift from discriminative reward models to generative reward models.
Related stories
arXiv AI
Jun 26
Joint Reward Modeling: Internalizing Chain-of-Thought for Efficient Visual Reward Models
arXiv:2602. 07533v2 Announce Type: replace Abstract: Reward models are critical for reinforcement learning from human feedback, as they determine the alignment quality and reliability of generative models.
arXiv AI
Jul 3
Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning
arXiv:2604. 02091v2 Announce Type: replace-cross Abstract: Rerankers play a pivotal role in refining retrieval results for Retrieval-Augmented Generation.
arXiv Machine Learning
Jul 21
Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
arXiv:2509. 02522v3 Announce Type: replace-cross Abstract: Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have empowered large language models (LLMs) to tackle challenging reasoning tasks such as mathematics and programming, however existing RLVR methods often suffer from sparse reward signals and unstable policy gradient updates inherent to RL-based approaches.
arXiv AI
Jun 9
Reasoning Arena: Trace Tournaments When Verifiable Rewards Fall Short
arXiv:2606. 09380v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a leading paradigm for improving the reasoning ability of large language models through outcome-based supervision.
arXiv AI
Jul 7
Adaptive Margin RLHF via Preference over Preferences
arXiv:2509. 22851v4 Announce Type: replace-cross Abstract: Margin-based optimization is fundamental to improving generalization and robustness in classification tasks.
arXiv AI
Jul 23
ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking
arXiv:2601. 06487v3 Announce Type: replace-cross Abstract: Reinforcement learning has substantially improved the performance of LLM agents on tasks with verifiable outcomes, but it still struggles on open-ended agent tasks with vast solution spaces (e.
arXiv Machine Learning
Jun 2
From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning
arXiv:2606. 01123v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) avoids explicit reward engineering by learning from pairwise human preference feedback.
arXiv AI
Aug 5
Don't Peek at the Answer: Outcome-Masked Group Relative Policy Optimization for Label-Free RLVR
arXiv:2608. 03119v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves LLM reasoning but typically relies on ground-truth (GT) answers, limiting scalability.
arXiv AI
Jun 2
Value-Free Policy Optimization via Reward Partitioning
arXiv:2506. 13702v4 Announce Type: replace-cross Abstract: Single-trajectory preference optimization methods learn from datasets of ((prompt, response, reward)) tuples, offering a practical alternative to pairwise preference learning by directly leveraging scalar feedback.
arXiv Machine Learning
Jun 10
Representation-Aware Advantage Estimation: Your Reward Model Provides More Than A Scalar Output
arXiv:2606. 10528v1 Announce Type: new Abstract: Current reinforcement learning from human feedback (RLHF) methods primarily rely on scalar rewards from a trained reward model (RM).
arXiv AI
Jul 7
MARS: Margin and Semantic-Aware Data Augmentation for Reward Modeling
arXiv:2602. 17658v3 Announce Type: replace-cross Abstract: Reward modeling is central to alignment pipelines such as RLHF, RLAIF, and PPO-based policy optimization, yet its reliability is constrained by limited and heterogeneous human preference data that are expensive to collect at scale.