The paper introduces a multi-reward reinforcement learning from internal feedback (RLIF) framework that splits the training signal into an answer-level reward from cluster voting and a completion-level reward from token-wise self-certainty. It uses GDPO-based normalization to balance reward scales and adds KL-Cov regularization to maintain exploration and avoid entropy collapse. Experiments on mathematical reasoning and code-generation tasks show the method improves stability and robustness compared to prior unsupervised RL approaches, approaching the performance of supervised RLVR methods.
arXiv:2608. 03119v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves LLM reasoning but typically relies on ground-truth (GT) answers, limiting scalability.
arXiv:2604. 02621v2 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) substantially improves the reasoning capabilities of language models, but most existing RL fine-tuning approaches rely entirely on ground-truth verifiable rewards and thus labeled datasets with verifiable answers.
Co‑RL is a multi‑agent reinforcement learning framework that trains several decoupled models without shared parameters, using rewards generated by their peers. By increasing cohort diversity—through heterogeneous model families, varying sizes, and rephrased training samples—Co‑RL reduces self‑reinforcing feedback loops, preserves behavioral diversity, and prevents training collapse. Across both text‑only and multimodal benchmarks, Co‑RL outperforms base models and prior label‑free methods, achieving gains of 3.0‑8.6% on seven text benchmarks and 2.3‑7.2% on four multimodal benchmarks, while matching or surpassing supervised approaches without any ground‑truth labels.
arXiv:2606. 04516v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) significantly advances LLM reasoning, yet it faces a dilemma: standard supervised scaling is throttled by high annotation costs, while unsupervised alternatives suffer from severe model collapse.
arXiv:2608.30632v1 Announce Type: cross Abstract: Reinforcement learning (RL), particularly RL with Verifiable Rewards (RLVR), has recently emerged as a central paradigm for enhancing large language...
arXiv:2602. 07832v3 Announce Type: replace-cross Abstract: Process rewards have been widely used in deep reinforcement learning to improve training efficiency, reduce variance, and prevent reward hacking.
arXiv:2610.18704v1 Announce Type: new Abstract: In this paper, we propose an extremely efficient, training-free method to extract token-level reward signals directly from an existing deep reward mode...
arXiv:2605. 21125v2 Announce Type: replace Abstract: Group Relative Policy Optimization (GRPO), a prominent algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improving the reasoning capabilities of large language models (LLMs).
The paper introduces Verifiable Process Rewards (VPR), a framework that uses symbolic or algorithmic task-solving oracles as action-level verifiers to provide dense rewards for intermediate actions in agentic reasoning tasks. VPR is applied to Sokoban, Sudoku, and Minesweeper, and improves policy performance over outcome-reward baselines. The authors also demonstrate VPR’s effectiveness in a tool‑use benchmark, highlighting its reliance on reliable task oracles and the difficulty of extending it to less structured environments.
arXiv:2607. 23802v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has driven recent progress in reasoning-oriented large language models (LLMs) by enabling large-scale optimization.
SIPO (Self‑Instructing Policy Optimization) unifies reinforcement learning with on‑policy self‑distillation by using a contrastive self‑teacher to generate token‑level credit signals. The method samples multiple rollouts per prompt, pairs each with a reference answer and its mistakes, and uses the difference in teacher log‑probabilities to provide dense feedback while still respecting the overall task reward. Experiments on reasoning and code‑generation benchmarks show that SIPO outperforms both RLVR and OPSD baselines without requiring an external teacher or extra generation steps.
arXiv:2607. 01181v1 Announce Type: cross Abstract: RL with verifiable rewards (RLVR) has emerged as a powerful paradigm for training LMs on tasks with well-defined success metrics, such as code generation and mathematical reasoning.