arXiv AI September 30, 2026 By Jiahua Yang, Zhiwei Yang, Xianpeng Zhang, Dongyu Chen, Xing Chen, Tianhuang Su, Haonan Lu, Quanlong Guan, Kai Tang, Chuangchuang Wang

Learn from the Gap: Differential-Aware Advantage Pruning with Adaptive Rollout Sampling for GRPO

Read the original on arXiv AI →

The paper introduces FastRL, a reinforcement learning framework designed to enhance the efficiency of Group Relative Policy Optimization (GRPO) and its variants. FastRL employs an advantage-aware pruning strategy that retains high-advantage trajectories while preserving gradient diversity, and an adaptive rollout sampling mechanism that adjusts sampling scale during training based on historical pruning data. Experiments show that FastRL can be integrated into GRPO, DAPO, and GSPO, yielding a 2.07× speedup on Geometry3K and GeoQA8K-R1V and a 1.64% accuracy improvement on visual reasoning benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 11

Parameter Exploration for RLVR via Variational Learning

arXiv:2608. 09805v1 Announce Type: cross Abstract: Exploration has been a focus of reinforcement learning research for a long time.

By Vatsal Venkatkrishna, Nico Daheim, Iryna Gurevych

llms reinforcement-learning

arXiv Machine Learning

Jun 16

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning

arXiv:2505. 09655v5 Announce Type: replace-cross Abstract: Post-training LLMs with Reinforcement Learning, specifically Group Relative Policy Optimization (GRPO), has emerged as a paradigm for enhancing mathematical reasoning.

By Xiwen Chen, Wenhui Zhu, Peijie Qiu, Xuanzhao Dong, Hao Wang, Haiyu Wu, Huayu Li, Aristeidis Sotiras, Yalin Wang, Abolfazl Razi

llms reinforcement-learning benchmarks safety

arXiv AI

Jul 7

R$^2$PO: Decoupling Rollout and Inference Policies for LLM Reasoning

arXiv:2601. 11960v3 Announce Type: replace-cross Abstract: Existing reinforcement learning methods for LLM reasoning implicitly assume that the policy generating training trajectories should coincide with the one producing inference responses.

By Jingchu Wang, Bingbing Xu, Yige Yuan, Dan Zhang, Bin Xie, Xiaoqian Sun, Huawei Shen

llms reinforcement-learning safety

arXiv AI

Jun 3

Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO

arXiv:2605. 30789v2 Announce Type: replace-cross Abstract: We identify a new dimension for enhancing rollout diversity in Group Relative Policy Optimization (GRPO) for LLMs.

By Yiming Ren, Yiran Xu, Zicheng Lin, Chufan Shi, Yukang Chen, Dingdong Wang, Tianhe Wu, Junjie Wang, Yujiu Yang, Yu Qiao, Ruihang Chu

llms benchmarks

arXiv AI

Jul 7

RSPO: Reward-Swap Policy Optimization for Multi-Turn LLM Agents

arXiv:2607. 04713v1 Announce Type: cross Abstract: Reinforcement learning holds significant potential for training large language models (LLMs) to handle multi-turn interactive tasks.

By Qiang Liu, Taian Guo, Ruizhi Qiao, Xing Sun

llms agents reinforcement-learning efficiency benchmarks safety

arXiv AI

Jun 2

RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning

arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).

By Yixiu Mao, Yun Qu, Qi Wang, Heming Zou, Xiangyang Ji

llms reinforcement-learning fine-tuning benchmarks safety

One email a morning, machine-written

Related stories

Parameter Exploration for RLVR via Variational Learning

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning

R$^2$PO: Decoupling Rollout and Inference Policies for LLM Reasoning

Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO

RSPO: Reward-Swap Policy Optimization for Multi-Turn LLM Agents

RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning