The paper investigates how advantage scale calibration can become imbalanced in group-relative optimization when rewards have low variance. It distinguishes two problematic low‑variance scenarios—sub‑resolution jitter and small but credible cardinal gaps—and introduces a three‑way calibration interface that links within‑group scale to reward‑branch strength, prompt‑level batch weight, and KL calibration. Building on this interface, the authors propose the Reward‑Resolution Protocol and MaxNorm‑AC to filter out sub‑resolution gaps and provide bounded recovery for credible gaps, achieving better performance than existing robust‑scale baselines across dense, MoE, math, and code reasoning tasks.
arXiv:2609.13866v1 Announce Type: new Abstract: Group-relative policy optimization (GRPO and descendants) can discard no-contrast rollout groups through dynamic sampling, while practical implementati...
arXiv:2606. 05800v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) often adopts GRPO-style group-relative updates, sampling multiple rollouts per prompt to construct normalized learning signals.
arXiv:2605. 17333v2 Announce Type: replace Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) typically samples multiple responses per prompt and assigns binary rewards based on individual correctness, yet the collective structure of the group output, specifically the distribution of errors, is largely discarded.
arXiv:2608. 01804v1 Announce Type: new Abstract: Post-training large language models (LLMs) via reinforcement learning (RL) has significantly advanced code generation capabilities.
Group Adaptive Clipping Policy Optimization (GAPO) is a plug‑in modification to GRPO methods that adapts the importance‑sampling clipping boundary based on rollout advantage. By allowing rollouts with larger learning signals to receive proportionally greater update headroom, GAPO addresses the limitation of fixed clipping that suppresses rare but informative rollouts. Experiments on Qwen and Llama models show that GAPO consistently improves Pass@1 and Pass@k on math reasoning and coding benchmarks where base model pass rates are low.
arXiv:2606. 27369v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) for training LLMs typically rely on ground-truth answers to assign rewards, limiting their applicability to tasks where the ground-truth solution is unknown.
arXiv:2606. 28707v1 Announce Type: new Abstract: Critic-free reinforcement learning with verifiable rewards (RLVR), exemplified by Group Relative Policy Optimization (GRPO), avoids training a value function (critic) and reduces memory and compute overhead relative to critic-based PPO pipelines for aligning large language models.
arXiv:2602. 05459v2 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) is typically benchmarked by the best tuned success rate of each method.
arXiv:2605. 17877v2 Announce Type: replace Abstract: A significant hurdle for current LLMs is the execution of complex, multi-stage tasks.
arXiv:2609.06107v1 Announce Type: new Abstract: Data policies for reinforcement learning with verifiable rewards (RLVR) determine which rollouts are used, how strongly they are weighted, and which do...
The paper introduces Adaptive Conformal Inference, an online calibration method that replaces static quantile estimation in hardware‑aware neural architecture search (NAS). By providing distribution‑free guarantees that at most a fraction δ of candidates are wrongly discarded, the method maintains steerable coverage across arbitrary sequences of reinforcement‑learning proposals. Experiments on three neural‑network families show that it tracks requested coverage levels within ~10⁻³ while pruning 25–50% of evaluations without accuracy loss, outperforming static calibration and a Gaussian‑process baseline.
arXiv:2609.39757v1 Announce Type: cross Abstract: Black-box distillation is a practical route for transferring capabilities from API-accessible large language models that expose only text outputs int...