arXiv Machine Learning
C-GRPO: Conformal Group Relative Policy Optimization
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
6d ago
When Does Closed-Loop Curriculum Adaptation Help GRPO? An Analysis on Countdown
The paper introduces CLOSEDLOOPGAUSSIAN, an adaptive curriculum strategy for Group Relative Policy Optimization (GRPO) that dynamically adjusts the pace and spread of an Easy-to-Hard Gaussian sampler based on probe estimates of per-difficulty accuracy. Unlike fixed easy-to-hard schedules, the method accelerates training when the model outpaces a reference curriculum, slows it down when it lags, and broadens sampling when earlier difficulties regress, thereby keeping training focused on learnable yet challenging tasks. Experiments on symbolic arithmetic reasoning demonstrate that CLOSEDLOOPGAUSSIAN outperforms vanilla GRPO and fixed open-loop curricula, especially in intermediate-difficulty regimes where group-relative learning signals are most informative.
arXiv AI
Jun 2
RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
arXiv Machine Learning
Jun 2
Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
arXiv:2605. 21125v2 Announce Type: replace Abstract: Group Relative Policy Optimization (GRPO), a prominent algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improving the reasoning capabilities of large language models (LLMs).
arXiv Machine Learning
Jul 30
Early Verdicts, Better Budgets: Sequential Adaptive Rollout Allocation for Compute-Efficient RLVR
arXiv:2607. 26253v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) is bottlenecked by rollout generation, yet many sampled prompts produce saturated groups (all responses correct or all incorrect) whose zero reward variance yields no policy-gradient signal.
arXiv AI
Oct 15
Learning with a Single Rollout via Pass@k Policy Optimization
The paper introduces Single‑Rollout Pass@k Policy Optimization (SR‑PPO), a reinforcement learning method that uses only one rollout per prompt to train language‑model agents. SR‑PPO combines Monte Carlo token‑level advantage estimation with Pass@k gradient reweighting and an online critic for baseline and weight estimation. Experiments show that SR‑PPO achieves stable learning and performance comparable to traditional Group Relative Policy Optimization (GRPO) on reasoning and logic benchmarks, even when off‑policy drift degrades standard Pass@1 PPO.
arXiv AI
Jun 2
Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
arXiv:2605. 12969v3 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) is one of the most widely adopted RLVR algorithms for post-training large language models on reasoning tasks.