arXiv AI

When Does Deep RL Beat Calibrated Baselines? A Benchmark Study on Adaptive Resource Control

arXiv:2605. 26418v2 Announce Type: replace-cross Abstract: A properly calibrated rule-based autoscaler can beat every one of six mainstream deep reinforcement learning (DRL) algorithms on cost across every workload we test - so when, if ever, does DRL actually help?

arXiv Machine Learning
Aug 26

WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation

arXiv:2608.24479v1 Announce Type: new Abstract: Massively parallel simulation changes the data regime in which off-policy reinforcement learning (RL) is trained, challenging stabilizers designed for...

By Zihao Wu, Hongyao Tang, Yi Ma, Huizhong Song, Pengyi Li, Yifu Yuan, Fei Ni, Jinyi Liu, Wei Wei, Jianrong Wang, Yan Zheng, Jianye Hao
arXiv AI
Jun 16

RollArt: Disaggregated Multi-Task Agentic RL Training at Scale

arXiv:2512. 22560v2 Announce Type: replace-cross Abstract: Agentic Reinforcement Learning (RL) trains LLMs through multi-turn interactions with environments, producing workloads that mix compute-bound prefill, bandwidth-bound decoding, CPU-heavy environment execution, and bursty reward evaluation.

By Wei Gao, Yuheng Zhao, Tianyuan Wu, Shaopan Xiong, Weixun Wang, Dakai An, Lunxi Cao, Dilxat Muhtar, Zichen Liu, Haizhou Zhao, Ju Huang, Siran Yang, Yongbin Li, Wenbo Su, Jiamang Wang, Lin Qu, Bo Zheng, Wei Wang
arXiv Machine Learning
Aug 12

TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling

arXiv:2608. 10402v1 Announce Type: new Abstract: Reinforcement learning (RL) for large language models is moving toward multi-turn agentic workloads, where rollout tasks repeatedly pause for external environments, resume with growing contexts, and finish at highly variable times.

By Yanyu Ren, Xizheng Wang, Xiao Liu, Bowen Lv, Hanchen Zhang, Shudan Zhang, Hanyu Lai, Shuai Wang, Li Chen, Dan Li, Jie Tang
arXiv Machine Learning
Sep 2

Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC

The paper introduces Solver-Gradient Guided Reinforcement Learning (SG‑RL), a method that augments standard RL with bounded gradients from a differentiable MPC solver to adapt cost‑function weights online. SG‑RL integrates solver‑gradient guidance into PPO through actor‑update scaling, policy loss, advantage estimation, and value‑function learning, achieving comparable or superior closed‑loop performance while requiring up to 70.6% fewer samples. Experiments on two autonomous racing platforms with intentional model mismatch demonstrate that SG‑RL outperforms both RL and gradient‑based policy learning baselines and generalizes zero‑shot to unseen environments.

By Baha Zarrouki, Arslan Thobani, Jasper Hoffmann, Mattia Piccinini, Rudolf Reiter, Felix Jahncke, S\'ebastien Gros, Davide Scaramuzza, Johannes Betz
arXiv Machine Learning
Sep 24

Learning to Remember: Attentive Reinforcement Learning for Edge Serverless Autoscaling

The paper introduces a stability‑aware autoscaling framework for edge serverless workloads that combines an Attention‑Enhanced Double‑Stacked LSTM with Proximal Policy Optimization to address temporal blindness in deep reinforcement learning. By weighting recent historical states non‑uniformly, the method suppresses high‑frequency jitter while preserving demand trends, outperforming single‑layer LSTM, static HPA, and KEDA baselines in latency reduction and stability. Experiments on two Kubernetes clusters with real Azure Functions traces show a ~67% reduction in P90 latency and improved adherence to a 50 ms hard SLO.

By Faraz Shaikh, Gianluca Reali, Mauro Femminella
arXiv Machine Learning
Jun 26

Reinforcement Learning without Ground-Truth Solutions can Improve LLMs

arXiv:2606. 27369v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) for training LLMs typically rely on ground-truth answers to assign rewards, limiting their applicability to tasks where the ground-truth solution is unknown.

By Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang, Xunpeng Huang, Kun Zhou, Tongtong Liang, Zhewei Yao, Yi-An Ma, Yuxiong He
arXiv Machine Learning
Aug 7

Hybrid-Adaptive Thread Tuning to Mitigate Simulation Execution Bottlenecks in High-Performance Reinforcement Learning Inference

arXiv:2608. 06025v1 Announce Type: new Abstract: In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, where workloads are highly dynamic and sensitive to runtime thread configurations.

By Jiming Su, Hantao Hua, Lujia Yin, Yiping Yao, Feng Zhu
arXiv Machine Learning
Jul 16

Where Should RL Post-Training Compute Go? Model Size, Search, Learning, and Feedback

arXiv:2607. 13389v1 Announce Type: new Abstract: Reinforcement Learning (RL) post-training is increasingly used to adapt foundation models for reasoning, planning, and feedback-driven robot-learning pipelines, but constrained post-training resources are often summarized by a single total FLOP budget.

By Patrick Wilhelm, Odej Kao