arXiv:2608. 15445v1 Announce Type: new Abstract: When a reward is correct on every training example yet consistent with more than one goal, a model can acquire an unintended one, a failure known as goal misgeneralization.
arXiv:2609.38850v1 Announce Type: new Abstract: Decision models such as Jev answer questions with probabilities, which are only useful if they are calibrated. Open-source reproductions rely on superv...
arXiv:2606. 24281v1 Announce Type: cross Abstract: Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success.
The paper introduces Circuit Reasoning Score (CRS), a data‑selection signal for reinforcement learning with verifiable rewards that uses attention‑head activity from a frozen base model to gauge reasoning engagement. CRS is computed in a single forward pass without reward labels or rollouts, and it shows that selecting problems with the lowest reasoning‑circuit engagement can outperform random selection on several medium‑difficulty benchmarks. However, the benefit depends on domain, model scale, and reward conditions, indicating that data selection in this setting is regime‑dependent rather than a fixed ranking of problem quality.
arXiv:2607.28077v2 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identica...
arXiv:2606. 01682v1 Announce Type: cross Abstract: Selecting the best response from multiple small-model samples using a stronger scorer is a simple inference-time strategy, but fails when the small model has already committed to incorrect reasoning paths.
arXiv:2607. 10139v1 Announce Type: cross Abstract: Selecting the correct answer from a pool of candidate reasoning chains is the engine of test-time scaling, yet the standard selectors each carry a cost: self-consistency inherits the errors of the single model it resamples, and trained reward models need labeled data and transfer poorly off-distribution.
arXiv:2605.11467v2 Announce Type: replace-cross Abstract: Reasoning models post-hoc rationalize answers they have already committed to internally, producing chains of *reasoning theater*: deliberativ...
arXiv:2610.34951v1 Announce Type: cross Abstract: Recent methods for Reinforcement Learning with Verifiable Rewards (RLVR) use rollout diversity to improve reasoning performance or training efficienc...
The paper introduces SIF, a reinforcement learning technique that uses selective internal feedback to improve language models’ reasoning without external verifiers. SIF constructs comparative rewards by comparing token‑level self‑certainty across multiple windows and peers, rewarding only when both voting levels agree, and applies sample and group‑level filtering to decide which responses influence policy updates. Experiments on three backbone models across 12 mathematical and scientific benchmarks show that SIF’s final checkpoints outperform the best‑validation checkpoints of Intuitor and Vigor, achieving a 4.71‑point accuracy gain on Qwen2.5‑Math‑1.5B and maintaining accuracy during continued optimization with minimal test‑time rollouts.
arXiv:2608. 03401v1 Announce Type: cross Abstract: Large language models often reason at length before answering, increasing cost and latency.
The paper introduces Step-wise Rubrics as Rewards (SRaR), a reinforcement learning framework that assigns rubric-based scores to individual reasoning steps in large language models. By normalizing step scores and combining them with outcome rewards through a decoupled advantage estimator, SRaR preserves criterion-level structure and mitigates issues like rewarding flawed steps or encouraging endless self‑correction. Experiments on 16K problems and six math benchmarks show that SRaR outperforms prior rubric methods, improving accuracy, increasing faithful reasoning rates, and reducing self‑correction loops.