arXiv Machine Learning
Jun 5
Reward Learning through Ranking Mean Squared Error
arXiv:2601. 09236v3 Announce Type: replace Abstract: Reward design remains a significant bottleneck in applying reinforcement learning (RL) to real-world problems.
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction
arXiv:2608. 06310v1 Announce Type: new Abstract: Recent advances in reward modeling show a paradigm shift from discriminative reward models to generative reward models.
Related stories
arXiv AI
4d ago
RankBuffer: Efficient Ranking-Based Rewards for Open-Ended Generation
arXiv:2609.36652v1 Announce Type: new Abstract: Open-ended generation lacks canonical answers, making pointwise rewards difficult to calibrate for group-based reinforcement learning. Directly ranking...
arXiv AI
Jun 26
Joint Reward Modeling: Internalizing Chain-of-Thought for Efficient Visual Reward Models
arXiv:2602. 07533v2 Announce Type: replace Abstract: Reward models are critical for reinforcement learning from human feedback, as they determine the alignment quality and reliability of generative models.
arXiv AI
Jul 3
Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning
arXiv:2604. 02091v2 Announce Type: replace-cross Abstract: Rerankers play a pivotal role in refining retrieval results for Retrieval-Augmented Generation.
arXiv Machine Learning
Jul 21
Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
arXiv:2509. 02522v3 Announce Type: replace-cross Abstract: Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have empowered large language models (LLMs) to tackle challenging reasoning tasks such as mathematics and programming, however existing RLVR methods often suffer from sparse reward signals and unstable policy gradient updates inherent to RL-based approaches.
arXiv Computation and Language
Sep 21
ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL
ArenaFlow is a hierarchical credit propagation framework designed to improve reinforcement learning for open-ended agent tasks. It uses tournament-based relative ranking to generate trajectory-level rewards and structured reflective evaluation to identify pivotal success steps, reusable strategy skills, and skill usage attribution. The framework propagates advantages to high-confidence steps and maintains a global skill memory, enabling more targeted optimization and reusable skill priors for future exploration.
arXiv AI
Jun 9
Reasoning Arena: Trace Tournaments When Verifiable Rewards Fall Short
arXiv:2606. 09380v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a leading paradigm for improving the reasoning ability of large language models through outcome-based supervision.
arXiv AI
Jul 7
Adaptive Margin RLHF via Preference over Preferences
arXiv:2509. 22851v4 Announce Type: replace-cross Abstract: Margin-based optimization is fundamental to improving generalization and robustness in classification tasks.
arXiv AI
Jul 23
ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking
arXiv:2601. 06487v3 Announce Type: replace-cross Abstract: Reinforcement learning has substantially improved the performance of LLM agents on tasks with verifiable outcomes, but it still struggles on open-ended agent tasks with vast solution spaces (e.
Hugging Face Trending Papers
Sep 20
RLVR$^{2}$: Reinforcement Learning with Verifiable Rubric-based Ranking
Reinforcement Learning with Verifiable Rewards (RLVR) is expanding from tasks with well-defined correctness signals, such as mathematics and code, toward multifaceted quality requirements specified by...
arXiv Machine Learning
Jun 2
From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning
arXiv:2606. 01123v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) avoids explicit reward engineering by learning from pairwise human preference feedback.
arXiv AI
Aug 5
Don't Peek at the Answer: Outcome-Masked Group Relative Policy Optimization for Label-Free RLVR
arXiv:2608. 03119v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves LLM reasoning but typically relies on ground-truth (GT) answers, limiting scalability.