arXiv Machine Learning September 14, 2026 By Mingyu Chen, Yefan Tao, Gerald Friedland, Xuezhou Zhang, Chris Kong

MInTRL: Off-policy Intervention can boost On-policy RL

Read the original on arXiv Machine Learning →

MInTRL (Minimal Intervention Reinforcement Learning) expands exploration in on-policy reinforcement learning by inserting sparse, local corrections into rollouts via a judge-intervention policy. These interventions replace erroneous suffixes and immediately return control to the main policy, allowing the agent to explore beyond its natural trajectory while maintaining on-policy data. The method uses a sequence-level advantage-regression objective, avoiding importance sampling, and demonstrates superior performance on math and code benchmarks compared to standard on-policy and off-policy baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

5d ago

Improving Online Reinforcement Learning via Bidirectional Behavior Prior Distillation

The paper introduces Bidirectional Behavior Prior Distillation (B2PD), a method that uses action‑value priors to train a conditional variational autoencoder for generating high‑value behavior support. These expert behavior priors are then distilled into the online reinforcement learning agent, reducing inefficient exploration and stabilizing policy updates. Experiments on state‑ and pixel‑based tasks show that B2PD improves sample efficiency while maintaining stable learning dynamics.

By Gong Gao, Xiao Lai, Jiaji Shen, Ning Jia, Xianhui Liu, Weidong Zhao

agents reinforcement-learning efficiency

arXiv Machine Learning

Aug 24

Decoupling Policy Extraction for Offline Reinforcement Learning

arXiv:2608.20909v1 Announce Type: new Abstract: Offline RL methods commonly jointly train the actor and critic, where the critic is used to guide the actor toward higher-value actions. This coupled l...

By Xuyao Lin, Yixiang Shan, Jinru Duan, Tao Yang, Xinyu Zhao, Runyu Lei, Yiming Zhao, Jiaxin Fan, Zongbao Feng, Peng Jia

rag reinforcement-learning

arXiv AI

Jun 2

RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning

arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).

By Yixiu Mao, Yun Qu, Qi Wang, Heming Zou, Xiangyang Ji

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Sep 15

Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating

arXiv:2603.11321v3 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards improves reasoning in large language models, yet on-policy learning often suffers from cold-st...

By Yuning Wu, Ke Wang, Haoran Liu, Chaoqun Jia, Devin Chen, Kai Wei

llms reinforcement-learning benchmarks

arXiv AI

Jul 7

LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL

arXiv:2607. 04412v1 Announce Type: new Abstract: Reinforcement learning (RL) for non-verifiable instruction following increasingly relies on LLM judges with prompt-specific rubrics as reward signals.

By Yujin Kim, Namgyu Ho, Sangmin Hwang, Joonkee Kim, Yongjin Yang, Sangmin Bae, Seungone Kim, Jaehun Jung, Se-Young Yun, Hwanjun Song

llms reinforcement-learning benchmarks

arXiv Machine Learning

Aug 6

Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning

arXiv:2608. 05080v1 Announce Type: new Abstract: Critic-free group-based reinforcement learning has become a scalable approach for post-training large language models.

By Zheyuan Zhang, Manqing Mao, Hong Wang, Zhuoer Wang, Samson Koelle, Jie Yuan, Yanjun Lin, James Feng, Nikki Lijing Kuang, Yanfang Ye, Wei Niu

llms reinforcement-learning

One email a morning, machine-written

Related stories

Improving Online Reinforcement Learning via Bidirectional Behavior Prior Distillation

Decoupling Policy Extraction for Offline Reinforcement Learning

RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning

Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating

LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL

Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning