arXiv AI
Jun 4
Rollout-Level Advantage-Prioritized Experience Replay for GRPO
arXiv:2606. 04560v1 Announce Type: cross Abstract: Reinforcement learning from verifiable rewards with GRPO is a standard approach for post-training reasoning LLMs.
Headroom-Drift Replay: A Primitive for Principled Replay Control in GRPO
Headroom-Drift Replay is a replay control primitive designed for GRPO that separates reuse into two decisions: Headroom ranks stored groups by remaining learning value, while Drift gates them by compatibility with the current policy. The method keeps the fresh on‑policy stream unchanged and adds no auxiliary generation or training machinery. Across mathematical reasoning, multimodal reasoning, and Agentic Search benchmarks, it outperforms naive replay and matches or exceeds broader replay methods on Avg Mean@32, delivering comparable quality at materially lower wall‑clock time in Agentic Search.
Related stories
arXiv Machine Learning
Jun 16
Replay What Matters: Off-Policy Replay for Efficient LLM Reinforcement Unlearning
arXiv:2606. 15333v1 Announce Type: cross Abstract: LLM unlearning has emerged as a cost-effective alternative to full retraining for removing hazardous knowledge from pretrained models while preserving general utility.
Hugging Face Trending Papers
Jul 21
REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning
Large-scale online reinforcement learning (RL) is the predominant means of eliciting advanced abilities including long-term reasoning and agentic tool use in large language models (LLMs). However, continuing to scale it across vast task domains of interest remains challenging in both computational infrastructure and cost, especially when considering RL as merely a one-off learning stage.
arXiv AI
Jul 23
REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning
arXiv:2607. 19450v1 Announce Type: cross Abstract: Large-scale online reinforcement learning (RL) is the predominant means of eliciting advanced abilities including long-term reasoning and agentic tool use in large language models (LLMs).
arXiv AI
Jul 7
R$^2$PO: Decoupling Rollout and Inference Policies for LLM Reasoning
arXiv:2601. 11960v3 Announce Type: replace-cross Abstract: Existing reinforcement learning methods for LLM reasoning implicitly assume that the policy generating training trajectories should coincide with the one producing inference responses.
arXiv AI
4d ago
LIMBO: Lifelong Inference-Time Memory and Budget Optimization for LLM Agents
arXiv:2609.14138v1 Announce Type: cross Abstract: As LLM agents become integrated into increasingly complex workflows, they must continually acquire new capabilities while retaining competence on pre...
arXiv AI
Aug 11
Parameter Exploration for RLVR via Variational Learning
arXiv:2608. 09805v1 Announce Type: cross Abstract: Exploration has been a focus of reinforcement learning research for a long time.
arXiv Machine Learning
Aug 11
Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training
arXiv:2604. 01499v2 Announce Type: replace Abstract: Evolution Strategies (ES) have emerged as a scalable gradient-free alternative to reinforcement learning based LLM fine-tuning, but it remains unclear whether comparable task performance implies comparable solutions in parameter space.
arXiv AI
Jun 2
RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
arXiv AI
Aug 25
EDGE: Experience-Distillation for Guided Exploration in Agentic Reinforcement Learning
arXiv:2608.21946v1 Announce Type: cross Abstract: Reinforcement learning with outcome-based objectives such as GRPO enables LLM-based agents to solve complex, long-horizon tasks, yet the reusable exp...
arXiv AI
Jun 12
ARROW: Augmented Replay for RObust World models
arXiv:2603. 11395v3 Announce Type: replace-cross Abstract: Continual reinforcement learning challenges agents to acquire new skills while retaining previously learned ones with the goal of improving performance in both past and future tasks.
arXiv AI
Jul 14
ARMOR: Stabilizing On-Policy LLM RL with Off-Policy Anchor Samples
arXiv:2607. 10481v1 Announce Type: cross Abstract: Reinforcement learning (RL) has significantly enhanced the reasoning capabilities of large language models (LLMs), yet the training process remains notoriously fragile.