arXiv Machine Learning July 13, 2026 By Elham Daneshmand, Majid Khadiv, Glen Berseth, Hsiu-Chin Lin

SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions

Read the original on arXiv Machine Learning →

arXiv:2607. 08925v1 Announce Type: new Abstract: Training reinforcement-learning agents directly on physical robots makes every fall costly, since a fall can damage the platform and cannot be undone like a simulator reset; the goal is therefore to minimize falls during training rather than trade them off against return, as constrained Markov decision process (MDP) formulations do.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 11

Directional-Clamp PPO

arXiv:2511. 02577v2 Announce Type: replace Abstract: Proximal Policy Optimization (PPO) is widely regarded as one of the most successful deep reinforcement learning algorithms, known for its robustness and effectiveness across a range of problems.

By Gilad Karpel, Ruida Zhou, Shoham Sabach, Mohammad Ghavamzadeh

reinforcement-learning

arXiv Machine Learning

Jun 9

Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy

arXiv:2606. 08779v1 Announce Type: new Abstract: Reinforcement Learning (RL) has emerged as a pivotal post-training paradigm, yet it frequently suffers from unpredictable sub-optimum performance or even training collapses.

By Jiashun Liu, Runze Liu, Xu Wan, Jing Liang, Hongyao Tang, Ling Pan

llms reinforcement-learning

arXiv Machine Learning

Jul 31

LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts

arXiv:2607. 27787v1 Announce Type: new Abstract: Reinforcement learning from verifiable rewards (RLVR) for mathematical reasoning suffers from a structural blind spot: on "cliff" prompts-those on which every sampled rollout in a group fails-the group-normalized advantage is identically zero, so GRPO produces no gradient on precisely the prompts at the frontier of the model's capability.

By Ken Ding

reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Aug 4

When Do Surrogate Updates Improve Decisions? A Local Theory of Trajectory-Wise Transfer

arXiv:2608. 01130v1 Announce Type: new Abstract: A broad range of models face the mismatch where they are updated through trajectory losses but are evaluated by downstream task reward.

By Yuyang Shen

llms

Hugging Face Trending Papers

Jul 30

LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts

Reinforcement learning from verifiable rewards (RLVR) for mathematical reasoning suffers from a structural blind spot: on "cliff" prompts-those on which every sampled rollout in a group fails-the group-normalized advantage is identically zero, so GRPO produces no gradient on precisely the prompts at the frontier of the model's capability. We introduce LoRA Scaffolded Policy Optimization (LSPO), a sampling-time mechanism that recovers this lost gradient.

reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 6

Retry Policy Gradients in Continuous Action Spaces

arXiv:2606. 05888v1 Announce Type: new Abstract: Retry-based objectives such as pass@K and max@K optimize the best return obtained from multiple sampled trajectories, and recent work has shown that they can promote exploration without explicit exploration bonuses.

By Soichiro Nishimori, Paavo Parmas

Related stories

Directional-Clamp PPO

Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy

LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts

When Do Surrogate Updates Improve Decisions? A Local Theory of Trajectory-Wise Transfer

LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts

Retry Policy Gradients in Continuous Action Spaces