The paper argues that the observed crossover in pass@$k$ performance between reinforcement learning with verifiable rewards (RLVR) and its base model is not always statistically confirmed. By constructing confidence bands across sampling budgets and performing power analyses, the authors show that many reported crossovers lack statistical support and that more prompts can improve detection more than more answers per prompt. They further demonstrate that RLVR’s effect on a prompt is better described as a conditional distribution (a Markov kernel) rather than a single curve, allowing predictions of crossovers in new data and clarifying how losses on difficult prompts can overturn early gains.
arXiv:2607. 20543v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) can improve one-sample accuracy while making a model worse under repeated sampling.
arXiv:2606. 15455v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a key approach for enhancing the reasoning abilities of large language models.
The paper introduces belief‑shift branching, a method for placing forks in tree‑structured reinforcement learning rollouts by identifying points where a model’s answer belief changes most. Unlike traditional structural or entropy‑based approaches, belief‑shift uses a probe, logit‑lens depth profile, or learned activation direction to locate pivots in the value curve, incurring minimal computational overhead. Experiments across multiple models and benchmarks show that belief‑shift forking consistently outperforms baseline methods, yielding significant gains in mathematics and code tasks.
arXiv:2609.01274v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves language-model reasoning, but how these gains relate to inference-time decoding and sear...
The paper introduces belief‑shift branching, a method for placing forks in tree‑structured reinforcement learning rollouts by detecting where a model’s answer belief changes most sharply. Unlike traditional fixed‑length or entropy‑based forking, this approach uses a lightweight probe or learned activation direction to identify pivots in the value curve, reducing unnecessary sampling. Experiments show that belief‑shift forking consistently outperforms baseline methods across multiple models and benchmarks, yielding significant gains in mathematics and code tasks.
arXiv:2510. 14807v3 Announce Type: replace Abstract: We revisit exploration collapse in reinforcement learning with verifiable rewards (RLVR), from the perspective of the \emph{candidate distribution} for next-token prediction.
arXiv:2605. 02909v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a powerful approach for improving the reasoning capabilities of large language models (LLMs).
arXiv:2606. 05932v1 Announce Type: cross Abstract: Reinforcement learning from verifiable rewards (RLVR) improves reasoning even when the reward signal is spurious -- assigning credit to the group-plurality answer rather than a ground-truth verifier.
The paper "Demystifying Reinforcement Learning Post-Training of Language Models" investigates how reinforcement learning (RL) post‑training enhances large language models (LLMs) for tasks such as reasoning, math, and coding. By isolating RL components in a controlled setting, the authors analyze how the base model’s prior distribution, reward granularity, prompt diversity, and model scale influence outcomes, using policy entropy to compare pre‑training, supervised fine‑tuning (SFT), and RL stages. The study clarifies the role of spurious rewards, the importance of the base model’s probability mass on desired behaviors, and how these factors interact to determine post‑training success, offering a practical primer for NLP researchers. "whyItMatters":"The work provides a clearer understanding of RL post‑training mechanics, helping researchers and practitioners effectively apply RL to improve LLM capabilities."
arXiv:2609.40115v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has become an important approach for improving reasoning during post-training. Recent work sugges...
arXiv:2606. 03238v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) makes large-scale post-training possible by replacing an underspecified human objective with learned and scalable proxies.
arXiv:2607. 07435v1 Announce Type: cross Abstract: Agents acting on our behalf in the real world (e.