arXiv Machine Learning
Jun 25
Minimax PAC Bounds for Learning in Exogenous Contextual MDPs
arXiv:2606. 25170v1 Announce Type: cross Abstract: We study PAC learning in tabular discounted Markov decision processes with exogenous i.
Optimal Sample Complexity of Stable Discounted Markov Decision Processes
Related stories
arXiv AI
Jul 7
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
arXiv:2605. 16103v2 Announce Type: replace Abstract: Q-learning is known to suffer from overestimation bias: because the Bellman update maximizes noisy or imperfect action-value estimates, positive errors can be selected and propagated, causing learned values to exceed the true optimal values.
arXiv Statistics ML
3d ago
Sharp Statistical Rates for Asynchronous TD Learning with Markovian Data
arXiv:2609.38880v1 Announce Type: new Abstract: We study the last iterate of standard tabular temporal-difference (TD) learning from a single trajectory of a finite Markov reward process. For discoun...
Hugging Face Trending Papers
4d ago
Sharp Statistical Rates for Asynchronous TD Learning with Markovian Data
We study the last iterate of standard tabular temporal-difference (TD) learning from a single trajectory of a finite Markov reward process. For discount factor $γ$, write $H=(1-γ)^{-1}$, and let $μ_{\...
arXiv Machine Learning
Jul 28
Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes
arXiv:2607. 22982v1 Announce Type: new Abstract: Natural Policy Gradient (NPG) is a well-established Reinforcement Learning algorithm that underlies widely used methods such as Trust Region Policy Optimization and Proximal Policy Optimization, both of which have demonstrated strong empirical success.
arXiv Machine Learning
Sep 23
Tight Sample Complexity Bounds for Entropic Best Policy Identification
The paper investigates best‑policy identification in finite‑horizon, risk‑sensitive reinforcement learning using the entropic risk measure. It identifies a gap between known lower bounds ≥ η(e^{|eta|H}) and upper bounds ≤ O(e^{2|eta|H}) for sample complexity, attributing the excess factor to loose concentration bounds for exponential utilities. By employing a forward‑model algorithm with KL‑based exploration bonuses and a novel stopping rule, the authors achieve a sample complexity that matches the lower bound, closing the previously open exponential gap.
arXiv Machine Learning
Aug 10
Robust Average-Reward Markov Decision Processes: Minimax-Optimal Learning via Plug-in Reductions
arXiv:2608. 06545v1 Announce Type: new Abstract: Distributionally robust Markov decision processes provide a principled framework for sequential decision making under model uncertainty.
arXiv Machine Learning
Aug 18
Near-Optimal Sample Complexity Bounds for Constrained Average-Reward MDPs
arXiv:2509. 16586v2 Announce Type: replace Abstract: Recent advances have significantly improved our understanding of the sample complexity of learning in average-reward Markov decision processes (AMDPs) under the generative model.
arXiv Machine Learning
Aug 4
Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning
arXiv:2608. 02034v1 Announce Type: new Abstract: Multi-step returns accelerate reward propagation in off-policy reinforcement learning, but couple the evaluation of each decision to the suboptimal logged actions that follow it, inducing a pessimistic bias that grows with the horizon.
arXiv Machine Learning
Jun 15
Lyapunov-Based Sample Complexity Analysis for Weakly-Coupled MDPs
arXiv:2606. 14095v1 Announce Type: new Abstract: We study the sample complexity of learning in average-reward weakly-coupled Markov decision processes (WCMDPs) and Restless Bandits (RBs) under a generative model.
arXiv Machine Learning
Aug 14
Decentralized Multi-Player Q-Learning in Episodic Markov Decision Processes with Information Asymmetry
arXiv:2608. 12753v1 Announce Type: new Abstract: We study decentralized multi-player reinforcement learning in episodic tabular Markov decision processes (MDPs) under three forms of information asymmetry: (A) unobserved actions with common rewards, (B) observed actions with independent rewards, and (C) unobserved actions with independent rewards.
arXiv Machine Learning
Sep 24
Limiting-Kernel Q($\lambda$): Bridging Short and Long Horizons
Limiting‑Kernel Q(λ) (LKQL) is an off‑policy value estimator that blends n‑step truncation with a long‑horizon approximation based on the limiting kernel. It maintains the computational efficiency of n‑step methods while improving policy evaluation accuracy, especially for long‑horizon tasks. The authors prove faster convergence of LKQL’s operator under aperiodicity and near‑on‑policy conditions, and demonstrate empirical gains on MuJoCo continuous‑control benchmarks.