SCARL (Sequence Confidence-Aligned Reinforcement Learning) addresses a misalignment between a reinforcement learning policy’s confidence and its reward signal in language-model reasoning. By reshaping each advantage with an alignment gap that reflects the difference between a response’s group-relative confidence and the outcome it certifies, SCARL drives responses with equal advantage toward equal confidence and mitigates entropy collapse. Experiments on Qwen3‑4B‑Base show a 2.4‑point lift in Pass@32 over DAPO and consistent improvements across multiple estimators and backbones while preserving policy entropy.
arXiv:2610.35059v1 Announce Type: cross Abstract: Large language models (LLMs) can produce plausible but incorrect answers when they are uncertain, which can mislead users and undermine model reliabi...
arXiv:2605. 17333v2 Announce Type: replace Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) typically samples multiple responses per prompt and assigns binary rewards based on individual correctness, yet the collective structure of the group output, specifically the distribution of errors, is largely discarded.
arXiv:2609.06107v1 Announce Type: new Abstract: Data policies for reinforcement learning with verifiable rewards (RLVR) determine which rollouts are used, how strongly they are weighted, and which do...
arXiv:2608. 00220v1 Announce Type: new Abstract: We show that on-policy reinforcement learning with verifiable rewards (RLVR) can improve the current objective while making successful behaviors for later objectives too rare to sample and reinforce.
The paper introduces Step-wise Rubrics as Rewards (SRaR), a reinforcement learning framework that assigns rubric-based scores to individual reasoning steps in large language models. By normalizing step scores and combining them with outcome rewards through a decoupled advantage estimator, SRaR preserves criterion-level structure and mitigates issues like rewarding flawed steps or encouraging endless self‑correction. Experiments on 16K problems and six math benchmarks show that SRaR outperforms prior rubric methods, improving accuracy, increasing faithful reasoning rates, and reducing self‑correction loops.
arXiv:2610.27012v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) trains a language model by sampling responses, scoring them with an automatic verifier, and upd...
The paper introduces Circuit Reasoning Score (CRS), a data‑selection signal for reinforcement learning with verifiable rewards that uses attention‑head activity from a frozen base model to gauge reasoning engagement. CRS is computed in a single forward pass without reward labels or rollouts, and it shows that selecting problems with the lowest reasoning‑circuit engagement can outperform random selection on several medium‑difficulty benchmarks. However, the benefit depends on domain, model scale, and reward conditions, indicating that data selection in this setting is regime‑dependent rather than a fixed ranking of problem quality.
arXiv:2607. 20543v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) can improve one-sample accuracy while making a model worse under repeated sampling.
arXiv:2609.36587v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a prominent approach for improving language-model performance on reasoning tasks using...
The paper introduces Residual Advantage (RA), a new guidance method for reinforcement learning with verifiable rewards (RLVR) that treats the teacher–student probability residual as a bounded one‑step reward, subtracts the student’s state value to form an advantage, and centers it within each response. RA preserves the verifier’s mean label while redistributing credit among response steps, and its extension CoRA updates the teacher model using verifier advantages to adapt guidance to the student’s attempts. Experiments with Qwen3 models show that RA combined with GRPO or REINFORCE++ consistently outperforms teacher‑only on‑policy distillation across three mathematical benchmarks, improving macro Avg@8 and Pass@8 scores, with CoRA providing additional gains.
arXiv:2606. 03234v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has become the dominant approach for improving mathematical reasoning in large language models, yet current methods reduce each correct rollout to a single reward bit, ignoring the geometric structure shared among their hidden states.
arXiv:2607. 27787v1 Announce Type: new Abstract: Reinforcement learning from verifiable rewards (RLVR) for mathematical reasoning suffers from a structural blind spot: on "cliff" prompts-those on which every sampled rollout in a group fails-the group-normalized advantage is identically zero, so GRPO produces no gradient on precisely the prompts at the frontier of the model's capability.