arXiv Machine Learning September 18, 2026 By Gong Gao, Xiao Lai, Jiaji Shen, Ning Jia, Xianhui Liu, Weidong Zhao

Improving Online Reinforcement Learning via Bidirectional Behavior Prior Distillation

Read the original on arXiv Machine Learning →

The paper introduces Bidirectional Behavior Prior Distillation (B2PD), a method that uses action‑value priors to train a conditional variational autoencoder for generating high‑value behavior support. These expert behavior priors are then distilled into the online reinforcement learning agent, reducing inefficient exploration and stabilizing policy updates. Experiments on state‑ and pixel‑based tasks show that B2PD improves sample efficiency while maintaining stable learning dynamics.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 24

Expert Behavior Prior Reinforcement Learning

arXiv:2607. 21302v1 Announce Type: new Abstract: Behavior prior reinforcement learning (BPRL) has emerged as a promising paradigm to improve sample efficiency in online reinforcement learning (RL) by leveraging policy priors derived from offline demonstrations.

By Gong Gao, Weidong Zhao, Xianhui Liu, Ning Jia

agents reinforcement-learning robotics benchmarks

arXiv Machine Learning

Aug 24

Decoupling Policy Extraction for Offline Reinforcement Learning

arXiv:2608.20909v1 Announce Type: new Abstract: Offline RL methods commonly jointly train the actor and critic, where the critic is used to guide the actor toward higher-value actions. This coupled l...

By Xuyao Lin, Yixiang Shan, Jinru Duan, Tao Yang, Xinyu Zhao, Runyu Lei, Yiming Zhao, Jiaxin Fan, Zongbao Feng, Peng Jia

rag reinforcement-learning

arXiv Machine Learning

Sep 14

MInTRL: Off-policy Intervention can boost On-policy RL

MInTRL (Minimal Intervention Reinforcement Learning) expands exploration in on-policy reinforcement learning by inserting sparse, local corrections into rollouts via a judge-intervention policy. These interventions replace erroneous suffixes and immediately return control to the main policy, allowing the agent to explore beyond its natural trajectory while maintaining on-policy data. The method uses a sequence-level advantage-regression objective, avoiding importance sampling, and demonstrates superior performance on math and code benchmarks compared to standard on-policy and off-policy baselines.

By Mingyu Chen, Yefan Tao, Gerald Friedland, Xuezhou Zhang, Chris Kong

reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 11

OGPO: Sample Efficient Full-Finetuning of Generative Control Policies

arXiv:2605. 03065v2 Announce Type: replace Abstract: Generative control policies (GCPs), such as diffusion- and flow-based control policies, have emerged as effective parameterizations for robot learning.

By Sarvesh Patil, Mitsuhiko Nakamoto, Manan Agarwal, Shashwat Saxena, Jesse Zhang, Giri Anantharaman, Cleah Winston, Chaoyi Pan, Douglas Chen, Nai-Chieh Huang, Zeynep Temel, Oliver Kroemer, Sergey Levine, Abhishek Gupta, Hongkai Dai, Paarth Shah, Max Simchowitz

diffusion reinforcement-learning robotics fine-tuning benchmarks

arXiv Machine Learning

Jun 9

Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy

arXiv:2606. 08779v1 Announce Type: new Abstract: Reinforcement Learning (RL) has emerged as a pivotal post-training paradigm, yet it frequently suffers from unpredictable sub-optimum performance or even training collapses.

By Jiashun Liu, Runze Liu, Xu Wan, Jing Liang, Hongyao Tang, Ling Pan

llms reinforcement-learning

arXiv Machine Learning

Jul 9

Latent Policy Steering through One-Step Flow Policies

arXiv:2603. 05296v2 Announce Type: replace-cross Abstract: Offline reinforcement learning (RL) allows robots to learn from offline datasets without risky exploration.

By Hokyun Im, Andrey Kolobov, Jianlong Fu, Youngwoon Lee

reinforcement-learning robotics efficiency benchmarks

One email a morning, machine-written

Related stories

Expert Behavior Prior Reinforcement Learning

Decoupling Policy Extraction for Offline Reinforcement Learning

MInTRL: Off-policy Intervention can boost On-policy RL

OGPO: Sample Efficient Full-Finetuning of Generative Control Policies

Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy

Latent Policy Steering through One-Step Flow Policies