arXiv:2509. 25004v2 Announce Type: replace Abstract: Online reinforcement learning with verifiable rewards (RLVR) has become an effective paradigm for improving the reasoning abilities of large language models, but most methods still optimize reasoning trajectories over the static problem set, wasting rollout budget on solved or overly difficult problems.
arXiv:2601. 18778v3 Announce Type: replace Abstract: RL methods for scaling large reasoning models stall on datasets with low initial success rates, and thus little training signal.
The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.
Recent advances in large language models (LLMs) have demonstrated that reinforcement fine-tuning of pretrained base models can lead to significant gains in reasoning performance at inference time. In this work, we theoretically analyze why reinforcement fine-tuning induces better reasoning ability than purely supervised fine-tuning (SFT) methods.
arXiv:2608. 01522v1 Announce Type: new Abstract: Teaching a language model a skill it has not mastered is obstructed by three recurring difficulties: training data is scarce, ground-truth reasoning traces are usually unavailable, and models often exhibit an apparent ceiling beyond which additional data yields no further improvement.
arXiv:2609.33149v2 Announce Type: replace Abstract: A common principle of effective learning is to practice material that is neither already mastered nor too difficult to permit progress. We ask how...
The paper investigates whether the high costs of training chain-of-thought reasoning models can be reduced through algorithmic design. It introduces an autocurriculum approach that lets the model select which problems to focus on during training, showing that this method provably improves both supervised fine‑tuning and reinforcement learning. For supervised fine‑tuning, autocurriculum requires exponentially fewer reasoning demonstrations by targeting prompts where the model struggles, while for reinforcement learning it decouples computational cost from the quality of the reference model, making the burn‑in cost nearly independent of target accuracy.
The paper introduces a method to enhance large language model (LLM) exploration in Reinforcement Learning with Verifiable Rewards (RLVR) by guiding the target model with partial reasoning trajectories from smaller, weaker language models. This weak-model guidance disrupts over‑confidence, preserves generative diversity, and mitigates entropy collapse without extra fine‑tuning or complex reward designs. Experiments on mathematical benchmarks show consistent improvements over vanilla RLVR, especially as the number of allowed attempts ($k$) increases, indicating broader reasoning coverage.
Cliff is a reward‑shaping method for reinforcement learning with verifiable rewards that identifies the first mistake in a language model’s reasoning process using an off‑the‑shelf teacher. By splitting each rollout into a correct prefix and an incorrect suffix, Cliff assigns token‑level advantages—positive for correct tokens and negative for the rest—providing fine‑grained supervision. Across 12 scenarios, Cliff outperforms on‑policy distillation by 15% and standard GRPO by 7%, even when the teacher is only modestly capable.
arXiv:2605. 28829v2 Announce Type: replace-cross Abstract: Competitive STEM examinations such as JEE and NEET require multi-step symbolic reasoning, precise numerical computation, and deep conceptual understanding across physics, chemistry, and mathematics.
arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.
arXiv:2609.38409v1 Announce Type: new Abstract: Recent progress in large language model reasoning has been driven by benchmarks and reinforcement learning environments with automatically verifiable r...