arXiv Machine Learning

WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation

Hugging Face Trending Papers

Aug 25

WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation

Massively parallel simulation changes the data regime in which off-policy reinforcement learning (RL) is trained, challenging stabilizers designed for data-limited replay. Through controlled experimen...

reinforcement-learning robotics efficiency benchmarks

arXiv AI

Jun 18

Spotlight: Synergizing Seed Exploration and Spot GPUs for DiT RL Post-Training

arXiv:2606. 19004v1 Announce Type: cross Abstract: Reinforcement learning (RL) post-training of Diffusion Transformers (DiTs) is prohibitively expensive, requiring thousands of high-end GPUs.

By Ruiqi Lai, Dakai An, Wei Gao, Ju Huang, Siran Yang, Jiamang Wang, Lin Qu, Dmitrii Ustiugov, Wei Wang

llms diffusion computer-vision reinforcement-learning

arXiv Machine Learning

Jun 16

Replay What Matters: Off-Policy Replay for Efficient LLM Reinforcement Unlearning

arXiv:2606. 15333v1 Announce Type: cross Abstract: LLM unlearning has emerged as a cost-effective alternative to full retraining for removing hazardous knowledge from pretrained models while preserving general utility.

By Zirui Pang, Chenlong Zhang, Haosheng Tan, Zhuoran Jin, Jiaheng Wei, Zixin Zhong

llms reinforcement-learning benchmarks

arXiv AI

Jul 14

EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents

arXiv:2607. 09773v1 Announce Type: new Abstract: Computer-use agents must solve long-horizon tasks through repeated interaction with partially observable, multimodal desktop environments.

By Mianqiu Huang, Taofeng Xue, Chong Peng, Jinrui Ding, Sicheng Fan, Jiale Hong, Yufei Gao, Xiaocheng Zhang, Linsen Guo, Xin Yang, Dengchang Zhao, Yuchen Xie, Peng Pei, Xunliang Xie, Xipeng Qiu

agents reinforcement-learning multimodal

arXiv AI

Jun 9

sGPO: Trading Inference FLOPs for Training Efficiency in RLVR

arXiv:2606. 08854v1 Announce Type: cross Abstract: Standard Reinforcement Learning with Verifiable Rewards (RLVR) training allocates a fixed rollout budget to every query, without regard for what each query's difficulty means for the current policy.

By Shivchander Sudalairaj, Kai Xu, Akash Srivastava, Giorgio Giannone

reinforcement-learning

arXiv Machine Learning

Aug 7

Hybrid-Adaptive Thread Tuning to Mitigate Simulation Execution Bottlenecks in High-Performance Reinforcement Learning Inference

arXiv:2608. 06025v1 Announce Type: new Abstract: In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, where workloads are highly dynamic and sensitive to runtime thread configurations.

By Jiming Su, Hantao Hua, Lujia Yin, Yiping Yao, Feng Zhu

reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Aug 12

Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks

arXiv:2608. 10357v1 Announce Type: cross Abstract: Long-horizon tool-using agents must reason over user goals, domain policies, tool calls, simulator state, and delayed verifiable rewards.

By Zelei Cheng, Amritansh Mishra, Sambit Sahu, William Campbell

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jul 22

Stale but Stable: Staleness-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning

arXiv:2607. 18722v1 Announce Type: new Abstract: Asynchronous reinforcement learning improves throughput by decoupling rollout generation from optimization, but staleness is an inevitable byproduct compounded by policy lag, engine delays, and mixture-of-experts routing.

By Junyao Yang, Yucheng Shi, Zongxia Li, Zhongzhi Li, Ruhan Wang, Xiangxin Zhou, Kishan Panaganti, Haitao Mi, Leowei Liang

reinforcement-learning efficiency

arXiv AI

Jul 9

Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning

arXiv:2607. 07508v1 Announce Type: cross Abstract: Reinforcement learning (RL) is becoming increasingly important for post-training large language models (LLMs).

By Zhenyu Hou, Yujiang Li, Jie Tang, Yuxiao Dong

llms agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jul 10

HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning

arXiv:2601. 22448v2 Announce Type: replace Abstract: RLVR has become a standard recipe for training LLMs on reasoning tasks with verifiable outcomes, but when rollout generation dominates the cost, efficiency hinges on which prompts are sampled and when.

By Weiqi Wang, Xin Liu, Binxuan Huang, Hejie Cui, Rongzhi Zhang, Changlong Yu, Shuowei Jin, Jingfeng Yang, Qingyu Yin, Zhengyang Wang, Zheng Li, Yifan Gao, Priyanka Nigam, Bing Yin, Lihong Li, Yangqiu Song

llms reinforcement-learning benchmarks

arXiv AI

Jul 14

ARMOR: Stabilizing On-Policy LLM RL with Off-Policy Anchor Samples

arXiv:2607. 10481v1 Announce Type: cross Abstract: Reinforcement learning (RL) has significantly enhanced the reasoning capabilities of large language models (LLMs), yet the training process remains notoriously fragile.

By Kexin Huang, Junkang Wu, Jinda Lu, Shuo Yang, Chiyu Ma, Jiancan Wu, Xiang Wang, Xiangnan He, Guoyin Wang, Jingren Zhou

llms reinforcement-learning benchmarks

Hugging Face Trending Papers

Jul 21

Stale but Stable: Staleness-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning

Asynchronous reinforcement learning improves throughput by decoupling rollout generation from optimization, but staleness is an inevitable byproduct compounded by policy lag, engine delays, and mixture-of-experts routing. From a trust-region perspective, this mismatch is critical: training-inference divergence governs approximation error in finite-horizon bounds, whereas PPO clipping only gates sampled outward updates, acting as a sampled surrogate rather than a full-policy constraint.

reinforcement-learning efficiency

Related stories