arXiv:2507.14843v5 Announce Type: replace
Abstract: Recent advances highlight Reinforcement Learning with Verifiable Rewards (RLVR) as a promising approach to enhancing LLM capabilities. However, it...
By Fang Wu, Weihao Xuan, Danny Xing, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, Yejin Choi
EliteReplay is a post‑training method that combines GRPO with a quality‑diversity archive to preserve diverse solutions in reinforcement learning. By collecting distinct strategies from a frozen base model before training and replaying them during training, the approach improves accuracy while maintaining a broader set of correct solutions. On text‑to‑SQL tasks, EliteReplay outperforms baselines, achieving higher pass@128 scores and up to 77% more distinct correct strategies, and it better handles ambiguous questions by covering multiple interpretations.
By Hanane Nour Moussa, Doeun Lee, Sachin Kumar
arXiv:2606. 15455v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a key approach for enhancing the reasoning abilities of large language models.
By Suqin Yuan, Jinkun Chen, Jiyang Zheng, Muyang Li, Lei Feng, Dadong Wang, Tao Xiang, Tongliang Liu, Bo An
The paper "Demystifying Reinforcement Learning Post-Training of Language Models" investigates how reinforcement learning (RL) post‑training enhances large language models (LLMs) for tasks such as reasoning, math, and coding. By isolating RL components in a controlled setting, the authors analyze how the base model’s prior distribution, reward granularity, prompt diversity, and model scale influence outcomes, using policy entropy to compare pre‑training, supervised fine‑tuning (SFT), and RL stages. The study clarifies the role of spurious rewards, the importance of the base model’s probability mass on desired behaviors, and how these factors interact to determine post‑training success, offering a practical primer for NLP researchers.
"whyItMatters":"The work provides a clearer understanding of RL post‑training mechanics, helping researchers and practitioners effectively apply RL to improve LLM capabilities."
By Donovan Clay, Saket Gollapudi, Sankar Harilal, Min Jang, Jacob Morrison, Sewoong Oh, Natasha Jaques
The paper introduces Exploration-Distillation (ExpDis), a method that separates exploration from optimization in reinforcement learning with verifiable rewards (RLVR) for language models. ExpDis trains explorer policies with a novelty bonus, filters their outputs for correctness, and distills the results into a student policy that is optimized without the novelty bonus. This iterative process enables aggressive exploration while preserving model quality, outperforming DAPO on multiple mathematical reasoning benchmarks and improving pass@$k$ scaling.
By Saif Punjwani, Micah Goldblum
arXiv:2605. 02909v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a powerful approach for improving the reasoning capabilities of large language models (LLMs).
By Kazuki Egashira, Mark Vero, Jasper Dekoninck, Florian E. Dorner, Robin Staab, Martin Vechev