arXiv AI
Aug 11
Parameter Exploration for RLVR via Variational Learning
arXiv:2608. 09805v1 Announce Type: cross Abstract: Exploration has been a focus of reinforcement learning research for a long time.
Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO
arXiv:2605. 30789v2 Announce Type: replace-cross Abstract: We identify a new dimension for enhancing rollout diversity in Group Relative Policy Optimization (GRPO) for LLMs.
Related stories
arXiv AI
Jul 7
R$^2$PO: Decoupling Rollout and Inference Policies for LLM Reasoning
arXiv:2601. 11960v3 Announce Type: replace-cross Abstract: Existing reinforcement learning methods for LLM reasoning implicitly assume that the policy generating training trajectories should coincide with the one producing inference responses.
arXiv Machine Learning
Jun 16
DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning
arXiv:2505. 09655v5 Announce Type: replace-cross Abstract: Post-training LLMs with Reinforcement Learning, specifically Group Relative Policy Optimization (GRPO), has emerged as a paradigm for enhancing mathematical reasoning.
arXiv AI
Jun 9
MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting
arXiv:2601. 09085v2 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) has become a standard approach for training mathematical reasoning models; however, its reliance on multiple completions per prompt makes training computationally expensive.
arXiv AI
Jun 17
Know Thy Reasoner: Not All Language Models Explore Alike
arXiv:2604. 10827v2 Announce Type: replace Abstract: Compute scaling for LLM reasoning trades off exploring solution approaches (\emph{breadth}) against refining promising ones (\emph{depth}), yet why a given trade-off works, and why it often fails to transfer across models, remains unclear.
arXiv AI
Jun 15
Rethinking the Trust Region in LLM Reinforcement Learning
arXiv:2602. 04879v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become a cornerstone for fine-tuning Large Language Models (LLMs), with Proximal Policy Optimization (PPO) serving as the de facto standard algorithm.
arXiv Machine Learning
Jul 13
Policy-based Tuning of Autoregressive Image Models with Instance- and Distribution-Level Rewards
arXiv:2603. 23086v2 Announce Type: replace Abstract: Autoregressive (AR) models are highly effective for image generation, yet their standard maximum-likelihood estimation training lacks direct optimization for sample quality and diversity.
arXiv Machine Learning
Jul 9
UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma
arXiv:2607. 06987v1 Announce Type: new Abstract: Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs).
arXiv Machine Learning
Jun 10
N-GRPO: Embedding-Level Neighbor Mixing for Enhanced Policy Optimization
arXiv:2606. 10768v1 Announce Type: new Abstract: The success of Large Language Models in mathematical reasoning relies heavily on the generation of diverse and valid solution paths during the rollout phase.
arXiv AI
Aug 7
Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging
arXiv:2608. 05541v1 Announce Type: new Abstract: Evolution Strategy (ES) is a promising alternative to gradient-based fine-tuning for resource-constrained Large Language Model (LLM) reasoning.
arXiv AI
Jun 2
RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
arXiv Machine Learning
Jun 25
ExTra: Exploratory Trajectory Optimization for Language Model Reinforcement Learning
arXiv:2606. 24994v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) for language-model reasoning can fail at both extremes of task difficulty: easy prompts often produce all-correct, low-diversity rollout groups with little gradient signal, while hard prompts can produce all-incorrect groups with no positive reward.