arXiv Machine Learning June 30, 2026 By Huaqing Zhang, Jingchu Gai, Juno Kim, Bingbin Liu, Andrej Risteski

When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon

Read the original on arXiv Machine Learning →

arXiv:2606. 30445v1 Announce Type: new Abstract: Online imitation learning (IL), particularly on-policy distillation, has emerged as a strong LLM post-training approach, often outperforming offline supervised fine-tuning (SFT).

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 1

Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback

arXiv:2606. 30923v1 Announce Type: cross Abstract: Imitation Learning is a natural framework for learning in sequential decision-making systems and has emerged as the dominant paradigm through which we understand language model training.

By Ved Sriraman, Peihan Liu, Daniel Hsu, Adam Block

llms fine-tuning efficiency

arXiv AI

Aug 3

When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning

arXiv:2607. 29617v1 Announce Type: cross Abstract: Imitation learning (IL)---training an agent to replicate expert behavior from demonstrations---underpins applications from robotics to language model training.

By Luca Viano, Antoine Moulin, Audrey Huang, Volkan Cevher, Philip Amortila, Dylan J. Foster

llms agents robotics efficiency

arXiv AI

Jul 7

The Three Regimes of Offline-to-Online Reinforcement Learning

arXiv:2510. 01460v4 Announce Type: replace-cross Abstract: Offline-to-online reinforcement learning (RL) has emerged as a practical paradigm that leverages offline datasets for pretraining and online interactions for fine-tuning.

By Lu Li, Tianwei Ni, Yihao Sun, Pierre-Luc Bacon

reinforcement-learning fine-tuning

arXiv AI

Jul 23

Leveraging Offline Supervision for Efficient and Generalizable Reinforcement Learning in Large-Scale Vision-Language-Action Models

arXiv:2607. 19399v1 Announce Type: cross Abstract: It is commonly observed that online reinforcement learning (RL) produces better-performing strategies than offline methods across a broad range of performance measures.

By Dmitriy Poyarkov, Aleksei Staroverov, Aleksandr I. Panov

reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Aug 12

Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning

arXiv:2608. 10473v1 Announce Type: cross Abstract: Offline-to-online (O2O) reinforcement learning aims to leverage policies pretrained on static datasets while improving them through online interaction.

By Daoyi Li, Yixian Zhang, Chao Yu, Wenbo Ding, Yu Wang

reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 2

FM-IRL: Flow-Matching for Reward Modeling and Policy Regularization in Reinforcement Learning

arXiv:2510. 09222v3 Announce Type: replace Abstract: Flow Matching (FM) has shown remarkable ability in modeling complex distributions and achieves strong performance in offline imitation learning for cloning expert behaviors.

By Zhenglin Wan, Jingxuan Wu, Xingrui Yu, Chubin Zhang, Mingcong Lei, Bo An, Ivor Tsang

diffusion reinforcement-learning

Related stories

Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback

When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning

The Three Regimes of Offline-to-Online Reinforcement Learning

Leveraging Offline Supervision for Efficient and Generalizable Reinforcement Learning in Large-Scale Vision-Language-Action Models

Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning

FM-IRL: Flow-Matching for Reward Modeling and Policy Regularization in Reinforcement Learning