PaceReasoner is a label‑free, test‑time reinforcement learning framework that improves short‑mode accuracy by leveraging a model’s long responses. It shares a trainable policy between long and short modes, using progressive, confidence‑weighted supervision and a behavior reward that shifts from encouraging short exploration to rewarding agreement with long answers. Across three backbones and four benchmarks, it raises short‑mode accuracy by 7.4–13.5 percentage points while keeping responses under 25 tokens on average.
arXiv:2610.17900v1 Announce Type: new Abstract: Extended chain-of-thought reasoning improves problem-solving performance but increases inference cost and latency, motivating more accurate answers wit...
arXiv:2511. 02130v2 Announce Type: replace-cross Abstract: We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens.
arXiv:2606. 18831v1 Announce Type: cross Abstract: Long-context reasoning is an essential capability for large language models, particularly when they are deployed as autonomous agents that must reason over lengthy trajectories.
arXiv:2507.21931v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) often produce plausible but poorly-calibrated answers, limiting their reliability on reasoning-intensive tasks....
The paper demonstrates that fine‑tuning reasoning models to predict their own confidence at intermediate steps—using only 600 self‑supervised examples—substantially improves inference efficiency. Without adding any explicit stopping or length penalties, the models generate up to 25 % fewer tokens while maintaining accuracy on mathematical, scientific, and coding benchmarks across several architectures. The study finds that confidence supervision preserves the models’ high‑level reasoning structure rather than merely suppressing specific behaviors.
The paper introduces Thinking-Free Policy Initialization (TFPI), a simple adaptation to Reinforcement Learning with Verifiable Reward (RLVR) that removes explicit thinking content during training and inference. TFPI bridges long Chain-of-Thought distillation with standard RLVR, reducing token usage and improving performance. Experiments show that TFPI accelerates RL convergence, raises performance ceilings, and yields token‑efficient reasoning models, enabling a 4B model to achieve 89.0% accuracy on AIME24 and 65.5% on LiveCodeBench with under 4K H20 hours.
arXiv:2609.37119v1 Announce Type: cross Abstract: Recent approaches to reinforcement learning (RL) post-training for large language models increasingly remove the critic to reduce training instabilit...
arXiv:2608. 03119v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves LLM reasoning but typically relies on ground-truth (GT) answers, limiting scalability.
arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.
Rationale-Guided Policy Optimization (RGPO) is a reinforcement‑learning framework that adaptively uses ground‑truth rationale information to scaffold a language model’s reasoning process. Instead of treating reference solutions as fixed imitation targets, RGPO temporarily incorporates rationales to help the model generate better responses, then reverts to unguided learning with higher‑reward, model‑generated solutions. Experiments in both language‑only and vision‑language tasks show that RGPO consistently outperforms RLVR baselines, with ablation studies confirming that adaptive rationale guidance is a key factor in its success.
The paper introduces SIF, a reinforcement learning technique that uses selective internal feedback to improve language models’ reasoning without external verifiers. SIF constructs comparative rewards by comparing token‑level self‑certainty across multiple windows and peers, rewarding only when both voting levels agree, and applies sample and group‑level filtering to decide which responses influence policy updates. Experiments on three backbone models across 12 mathematical and scientific benchmarks show that SIF’s final checkpoints outperform the best‑validation checkpoints of Intuitor and Vigor, achieving a 4.71‑point accuracy gain on Qwen2.5‑Math‑1.5B and maintaining accuracy during continued optimization with minimal test‑time rollouts.
arXiv:2602. 06960v3 Announce Type: replace-cross Abstract: Large reasoning models achieve strong performance by scaling inference-time chain-of-thought, but this paradigm suffers from quadratic cost, context length limits, and degraded reasoning due to lost-in-the-middle effects.