The paper extends theoretical guarantees for offline reinforcement learning to parameterized policy classes over large or continuous action spaces, overcoming limitations of prior algorithms that required finite action sets and state‑wise mirror descent. By addressing contextual coupling and linking mirror descent to natural policy gradients, the authors provide new analyses, algorithmic insights, and a surprising connection between offline RL and imitation learning.
By Xiang Li, Yuheng Zhang, Nan Jiang
arXiv:2607. 07859v1 Announce Type: new Abstract: Reinforcement learning (RL) research has increasingly shifted focus towards alignment, ensuring agents learn behaviors adhering to human values.
By Benjamin Poole, Minwoo Lee
arXiv:2607. 29617v1 Announce Type: cross Abstract: Imitation learning (IL)---training an agent to replicate expert behavior from demonstrations---underpins applications from robotics to language model training.
By Luca Viano, Antoine Moulin, Audrey Huang, Volkan Cevher, Philip Amortila, Dylan J. Foster
arXiv:2506. 06793v2 Announce Type: replace-cross Abstract: Reward assignment from scarce demonstrations is a key challenge in both offline and online imitation learning.
By Zixuan Dong, Yumi Omori, Keith Ross
arXiv:2606. 09758v1 Announce Type: cross Abstract: Parametric imitation learning via behavior cloning can suffer from poor generalization to out-of-distribution states due to compounding errors during deployment.
By Quinn Pfeifer, Ethan Pronovost, Paarth Shah, Khimya Khetarpal, Siddhartha Srinivasa, Abhishek Gupta
SynIL is a new framework for offline imitation learning that automatically assesses the quality of demonstration data without requiring labels. It uses motor synergy—a low‑dimensional coordinated movement pattern linked to proficiency—to generate dense, transition‑level reward signals through self‑supervised reward regression. Experiments on D4RL locomotion and Robomimic manipulation datasets show that synergy‑derived rewards align well with true rewards and that SynIL outperforms Behavior Cloning and rivals or surpasses offline reinforcement learning in sparse‑reward scenarios.
By Yuto Tanaka, Kyo Kutsuzawa, Martina Doku, Dai Owaki, Mitsuhiro Hayashibe
arXiv:2512. 16861v2 Announce Type: replace-cross Abstract: Long-horizon manipulation has been a long-standing challenge in the robotics community.
By Zihan Zhou, Animesh Garg, Ajay Mandlekar, Caelan Garrett
arXiv:2606. 02194v1 Announce Type: new Abstract: Distilling expert demonstration data into large generative models using behavioral cloning is a scalable approach to learning capable policies for robotic control, particularly for dexterous manipulation.
By Christian Scherer, Joe Watson, Theo Gruner, Daniel Palenicek, Ingmar Posner, Jan Peters
arXiv:2607. 01225v1 Announce Type: cross Abstract: Prior work on imitation learning from suboptimal demonstrations typically relies on compressed supervision signals such as confidence estimates, discriminator scores, or importance weights.
By Chih-Han Yang, Dai-Jie Wu, Yun-Ping Huang, Ping-Chun Hsieh, Kenneth Marino, Shao-Hua Sun
arXiv:2606. 01238v1 Announce Type: cross Abstract: While diffusion-based policies have impressive performance and expressivity, their long offline training slows down the data collection and policy deployment loop.
By Raghav Mishra, Ian R. Manchester
EVOL is a framework that uses a knowledge‑tracing simulator to generate per‑learner expert demonstrations via evolutionary search, and then trains a deployment‑free policy that distills these demonstrations into a feed‑forward learner. The method employs an asymmetric actor‑critic architecture, where the actor plans blindly as in real deployment while the critic has access to privileged simulator state during training. Across three educational datasets and varying path lengths, EVOL outperforms eight baselines and shows that the quality of evolutionary experts, rather than the imitation objective, drives final performance.
By Geonwoo Bang, Dongho Kim, Moohong Min
ASTEROID is a student–teacher supervised learning framework that learns efficient test‑time exploration in partially observable sequential decision problems. By training a history‑conditioned student on on‑policy contexts while a privileged expert provides action labels conditioned on that history, the student learns to search while uncertain and exploit once the hidden state is revealed. Across eleven diverse environments, ASTEROID achieves roughly twice the returns under fixed exploration budgets, uses about 100× fewer interactions than reinforcement learning baselines, and transfers well to vision‑denied real‑world robot manipulation, improving success by +40% over standard distillation.
By Sriyash Poddar, Yanda Bao, Jacob Krantz, Matthew Chang, Xavier Puig, Roozbeh Mottaghi, Natasha Jaques, Abhishek Gupta