arXiv Machine Learning

The Sample Complexity of Policy Learning with Mu-Resets

arXiv:2608. 07772v1 Announce Type: new Abstract: We study policy-based reinforcement learning under the $\mu$-resets interaction protocol of Kakade and Langford [KL02].

arXiv Machine Learning
Sep 23

Tight Sample Complexity Bounds for Entropic Best Policy Identification

The paper investigates best‑policy identification in finite‑horizon, risk‑sensitive reinforcement learning using the entropic risk measure. It identifies a gap between known lower bounds ≥ η(e^{|eta|H}) and upper bounds ≤ O(e^{2|eta|H}) for sample complexity, attributing the excess factor to loose concentration bounds for exponential utilities. By employing a forward‑model algorithm with KL‑based exploration bonuses and a novel stopping rule, the authors achieve a sample complexity that matches the lower bound, closing the previously open exponential gap.

By Amer Essakine, Claire Vernade
arXiv Machine Learning
Jun 16

Learning Policy from a Single Trajectory in Average-Reward Markov Decision Process

arXiv:2606. 16729v1 Announce Type: new Abstract: While there is an extensive body of work characterizing the sample complexity of discounted cumulative-reward MDPs, finite sample analyses for average-reward MDPs have been limited, and most existing works rely on restrictive assumptions such as ergodicity or access to a generative model.

By Jongmin Lee, Ernest K. Ryu, Vaneet Aggarwal
arXiv AI
Oct 12

Sampling-Based Safe Reinforcement Learning

Sampling-Based Safe Reinforcement Learning (SBSRL) is a model‑based RL algorithm that enforces safety by jointly constraining a finite set of dynamics samples, approximating a worst‑case optimization over uncertain dynamics. It introduces an exploration strategy that limits epistemic uncertainty, removing the need for explicit exploration bonuses. The authors provide high‑probability safety guarantees and a finite‑time sample complexity bound, and demonstrate safe, efficient exploration in both simulation and real robotic hardware, with extensions to deep‑ensemble implementations for high‑dimensional continuous control.

By Luca Vignola, Bruce D. Lee, Manish Prajapat, Manuel Wendl, Melanie Zeilinger, Andreas Krause, Yarden As