The paper introduces Step-wise Rubrics as Rewards (SRaR), a reinforcement learning framework that assigns rubric-based scores to individual reasoning steps in large language models. By normalizing step scores and combining them with outcome rewards through a decoupled advantage estimator, SRaR preserves criterion-level structure and mitigates issues like rewarding flawed steps or encouraging endless self‑correction. Experiments on 16K problems and six math benchmarks show that SRaR outperforms prior rubric methods, improving accuracy, increasing faithful reasoning rates, and reducing self‑correction loops.
arXiv:2605. 03862v4 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards has become a common way to improve explicit reasoning in large language models, but final-answer correctness alone does not reveal whether the reasoning trace is faithful, reliable, or useful to the model that consumes it.
Rubric Curriculum RL (RcRL) is a reinforcement learning approach that leverages the generation‑verification gap in non‑verifiable domains by using pairwise preferences over rollouts and a curriculum over rubric criteria. Unlike traditional methods that rely on absolute scoring and stationary judges, RcRL provides a more discriminative, non‑stationary signal, allowing policies to maintain output entropy and continue improving beyond 1,000 training steps. In experiments on creative writing and HealthBench, RcRL outperforms baseline models, achieving a 70.5% human win rate and significant accuracy gains over the base model.
Cliff is a reward‑shaping method for reinforcement learning with verifiable rewards that identifies the first mistake in a language model’s reasoning process using an off‑the‑shelf teacher. By splitting each rollout into a correct prefix and an incorrect suffix, Cliff assigns token‑level advantages—positive for correct tokens and negative for the rest—providing fine‑grained supervision. Across 12 scenarios, Cliff outperforms on‑policy distillation by 15% and standard GRPO by 7%, even when the teacher is only modestly capable.
arXiv:2610.34951v1 Announce Type: cross Abstract: Recent methods for Reinforcement Learning with Verifiable Rewards (RLVR) use rollout diversity to improve reasoning performance or training efficienc...
arXiv:2604.02341v2 Announce Type: replace-cross Abstract: Mathematical reasoning in large language models has improved substantially with reinforcement learning using verifiable rewards, where final...
CRoS (Causality Reward on Scrambled CoTs) is a new causality metric for chain-of-thought reasoning that perturbs the order of reasoning steps while preserving their content, avoiding the confounding effects of traditional perturbations. On Reasoning Gym with Gemma‑3‑4B, CRoS improves out‑of‑domain accuracy (46.8% vs. 43.8%), yields higher CoT quality (average CoT‑Pass@K 43.4% vs. 41.4%), and enhances CoT faithfulness (hint acknowledgment 34.0% vs. 20.2%).
arXiv:2608. 05643v1 Announce Type: new Abstract: Test-time scaling improves LLM reasoning by using additional inference compute, but wider sampling alone can suffer from diminishing returns: new rollouts often repeat existing answer patterns instead of adding useful reasoning diversity.
arXiv:2603. 05659v3 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) and Rubrics as Rewards (RaR) have driven strong gains in domains with clear correctness signals and even in subjective domains by synthesizing evaluation criteria from ideal reference answers.
The paper introduces Rubric Dropout, a technique that randomly removes a subset of rubric criteria during training to prevent reinforcement learning agents from exploiting the rubric as a proxy for quality. By resampling which criteria are rewarded at each step and ensuring all responses to a prompt are evaluated on the same subset, the method reduces reward hacking and improves out‑of‑distribution performance on medical and science benchmarks. Block Dropout further groups semantically similar criteria to avoid near‑duplicate rewards, yielding additional gains.
arXiv:2606. 19327v1 Announce Type: new Abstract: Post-training of reasoning language models is commonly driven by supervised distillation and reinforcement learning with verifiable rewards.
arXiv:2607. 11506v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) optimizes LLMs using sparse verifiable final-answer rewards.
Post-training of reasoning language models is commonly driven by supervised distillation and reinforcement learning with verifiable rewards. Distillation often relies on chain-of-thought annotations that are expensive to obtain and may themselves be noisy, incomplete, or partially incorrect; even when the final solution is correct, an imperfect rationale can interfere with learning.