arXiv Machine Learning
Interleaved Projected Gradient Descent for Safe Imitation Learning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
4d ago
The Price of Failure Risk: Clipped Barrier Policy Gradients with Convex Regularization
The paper introduces a clipped barrier policy gradient method that incorporates a convex regularizer to manage failure risk in budget-constrained operations. By adding a barrier that grows as the budget depletes and clipping rewards below a threshold, the approach ensures that the learned policy maximizes utility while keeping risk within a bounded margin. The authors prove exponential convergence of the training dynamics under convex regularization and demonstrate the method on several benchmark tasks.
arXiv Machine Learning
Sep 21
Provably Optimal Reinforcement Learning under Safety Filtering
The paper proves that using a permissive safety filter in reinforcement learning does not compromise asymptotic performance. By formalizing safety through a safety‑critical Markov decision process and a filtered MDP, the authors show that optimal policies in the filtered MDP achieve the same return as the best safe policy in the original setting. Experiments on Safety Gymnasium confirm zero violations during training and performance that matches or exceeds unfiltered baselines.
arXiv AI
Oct 15
Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning
arXiv:2610.17709v1 Announce Type: cross Abstract: Imitation learning typically treats every demonstration state as equally important, optimizing a behavior-cloning objective uniformly across the data...
arXiv Machine Learning
Sep 30
Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
The paper introduces Dually Regularized AIL, a model‑free algorithm for adversarial imitation learning that jointly applies KL policy regularization and a quadratic reward penalty based on expert and learner occupancies. It proves fast convergence rates, achieving a ×O(1/K+1/N) bound on the regularized imitation gap in finite‑horizon MDPs with general function approximation, and establishes the first algorithm to attain ×O(1/ε) sample complexity in both expert demonstrations and online interactions for this regularized objective.
arXiv AI
Sep 1
BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learning
BCPPO is a new variant of Proximal Policy Optimization that uses Bachelier-inspired cost‑prediction networks to generate a smooth penalty based on disagreement among critics. The method keeps temporal‑difference learning unchanged, applies a saturation‑aware controller to manage cost penalties, and deploys only the policy network. Across extensive experiments, BCPPO outperforms comparators in achieving higher mean returns while maintaining lower or comparable CVaR in all tested tasks.
arXiv AI
Oct 9
Higher-Order Action Supervision Makes A Strong Policy Class
The paper proposes a new loss scheme that simultaneously supervises zeroth- and first-order actions to improve policy performance and control robustness in data-driven decision-making methods such as imitation learning and reinforcement learning. The approach provides formal theoretical guarantees, works with any off-the-shelf policy model, and can be added as a lightweight plug‑and‑play module to existing offline RL frameworks. Experiments on OGBench and D4RL show significant gains in performance, robustness, and out‑of‑distribution generalization, especially in low‑data settings.