The Flow has not summarised this story yet — read it at arXiv Computation and Language.
SCARL (Sequence Confidence-Aligned Reinforcement Learning) addresses a misalignment between a reinforcement learning policy’s confidence and its reward signal in language-model reasoning. By reshaping each advantage with an alignment gap that reflects the difference between a response’s group-relative confidence and the outcome it certifies, SCARL drives responses with equal advantage toward equal confidence and mitigates entropy collapse. Experiments on Qwen3‑4B‑Base show a 2.4‑point lift in Pass@32 over DAPO and consistent improvements across multiple estimators and backbones while preserving policy entropy.
arXiv:2609.11149v3 Announce Type: replace-cross Abstract: How fast does a language model degrade when trained on its own outputs? Theory traces it to gradually accumulating errors, while experiments...
arXiv:2606. 15455v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a key approach for enhancing the reasoning abilities of large language models.
arXiv:2605. 17333v2 Announce Type: replace Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) typically samples multiple responses per prompt and assigns binary rewards based on individual correctness, yet the collective structure of the group output, specifically the distribution of errors, is largely discarded.
The paper critiques the use of pass@$k$ as a sole metric for evaluating reinforcement‑learning post‑training on language models, showing that it ignores diversity in model outputs. Experiments on Qwen2.5‑1.5B‑Instruct trained with Group Relative Policy Optimization (GRPO) and rejection‑sampling fine‑tuning (RFT) reveal that these methods shift token‑level, answer‑level, and unique‑answer diversity in opposite directions, yet pass@$k$ fails to capture these differences. Even when restricting to verifier‑correct completions, diversity gaps persist, and pass@$k$ does not consistently identify the better performing arm on hard problem subsets, indicating that the metric can mask true capability changes.
arXiv:2510. 11686v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) promises to expand the capabilities of language models, but it is unclear if current RL techniques promote the discovery of novel behaviors, or simply sharpen those already present in the base model.