arXiv Machine Learning
Jun 25
Minimax PAC Bounds for Learning in Exogenous Contextual MDPs
arXiv:2606. 25170v1 Announce Type: cross Abstract: We study PAC learning in tabular discounted Markov decision processes with exogenous i.
Second-Order Smooth Planning with Optimal-Transport Bellman Smoothing
Related stories
Hugging Face Trending Papers
Aug 20
Unregularized Convergence of Single-Loop, Entropy-Regularized Natural Actor-Critic
While entropy regularization is widely used to stabilize and accelerate Natural Policy Gradient methods, its ability to yield faster convergence rates for the unregularized objective remains underexplored. Existing analyses often rely on double-loop architectures and invoke a linear entropy penalty.
arXiv Machine Learning
Aug 21
Unregularized Convergence of Single-Loop, Entropy-Regularized Natural Actor-Critic
arXiv:2608. 19587v1 Announce Type: new Abstract: While entropy regularization is widely used to stabilize and accelerate Natural Policy Gradient methods, its ability to yield faster convergence rates for the unregularized objective remains underexplored.
arXiv Machine Learning
Sep 17
A Convergence Framework for Deep $V$-Learning: Error Propagation and Sharp Action-Gap Bounds
The paper presents a convergence framework for deep $V$‑learning over a finite horizon $H$, deriving explicit bounds on policy loss by decomposing the Bellman update error into six residuals. It shows how $L^s$ concentrability controls expected $L^1$ loss, quantifies the impact of shared sampling across horizon levels, and provides optimal and near‑optimal sample allocations for statistical error rates. The work also establishes sharp action‑gap bounds under a margin condition, transfers optimal‑gap results to frozen‑iterate gaps, and offers consistency guarantees for generative‑reset approximate‑ERM procedures with exact action scores.
arXiv Machine Learning
Sep 22
Optimal Sample Complexity of Stable Discounted Markov Decision Processes
arXiv:2302.07477v4 Announce Type: replace Abstract: We study the optimal sample complexity of tabular reinforcement learning for infinite-horizon discounted Markov decision processes. The unrestricte...
Hugging Face Trending Papers
Aug 24
SGHA: A Single-Loop Fully First-Order Algorithm for Nonconvex-Strongly-Convex Bilevel Optimization
In this work, we study the oracle complexity of finding an $ε$-stationary point for nonconvex-strongly-convex (NC-SC) bilevel optimization using only first-order oracles. Existing methods achieving th...
arXiv Machine Learning
Jul 28
Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes
arXiv:2607. 22982v1 Announce Type: new Abstract: Natural Policy Gradient (NPG) is a well-established Reinforcement Learning algorithm that underlies widely used methods such as Trust Region Policy Optimization and Proximal Policy Optimization, both of which have demonstrated strong empirical success.
arXiv Machine Learning
2d ago
Fast Regularized Policy Mirror Descent with One-Step TD Updates
arXiv:2609.39837v1 Announce Type: new Abstract: Policy mirror descent (PMD) enjoys fast convergence in regularized Markov decision processes (MDPs), but existing guarantees often rely on exact or inc...
arXiv Machine Learning
Jun 4
Policy Gradient for Continuous-Time Robust Markov Decision Processes
arXiv:2606. 04335v1 Announce Type: new Abstract: The framework of robust Markov decision processes (RMDPs) allows the design of reinforcement learning agents that satisfy performance guarantees under worst-case transition dynamics.
arXiv Machine Learning
Sep 23
Tight Sample Complexity Bounds for Entropic Best Policy Identification
The paper investigates best‑policy identification in finite‑horizon, risk‑sensitive reinforcement learning using the entropic risk measure. It identifies a gap between known lower bounds ≥ η(e^{|eta|H}) and upper bounds ≤ O(e^{2|eta|H}) for sample complexity, attributing the excess factor to loose concentration bounds for exponential utilities. By employing a forward‑model algorithm with KL‑based exploration bonuses and a novel stopping rule, the authors achieve a sample complexity that matches the lower bound, closing the previously open exponential gap.
arXiv Machine Learning
Aug 18
Near-Optimal Sample Complexity Bounds for Constrained Average-Reward MDPs
arXiv:2509. 16586v2 Announce Type: replace Abstract: Recent advances have significantly improved our understanding of the sample complexity of learning in average-reward Markov decision processes (AMDPs) under the generative model.
arXiv Machine Learning
Sep 1
Mirror Descent Linearized Augmented Lagrangian Methods for Nonconvex Constrained Stochastic Zeroth-Order Optimization
arXiv:2504.09409v3 Announce Type: replace-cross Abstract: In this paper, we study nonconvex constrained stochastic zeroth-order optimization problems with exact constraints and stochastic objective e...