arXiv Machine Learning June 3, 2026 By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

Data- and Variance-dependent Regret Bounds for Online Tabular MDPs

Read the original on arXiv Machine Learning →

arXiv:2602. 01903v2 Announce Type: replace Abstract: This work studies online episodic tabular Markov decision processes (MDPs) with known transitions and develops best-of-both-worlds algorithms that achieve refined data-dependent regret bounds in the adversarial regime and variance-dependent regret bounds in the stochastic regime.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 1

Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions

arXiv:2606. 31769v1 Announce Type: new Abstract: We study policy optimization for online episodic tabular Markov decision processes with unknown transition kernels, aiming for best-of-both-worlds guarantees together with data-dependent regret bounds.

By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

reinforcement-learning safety

arXiv Machine Learning

Jun 3

Online Learning with Gradient-Variation Interval Regret

arXiv:2606. 03831v1 Announce Type: new Abstract: This paper investigates non-stationary online learning using the metric of interval regret, which requires an online algorithm to perform well over every time interval.

By Yan-Feng Xie, Shuche Wang, Peng Zhao, Zhi-Hua Zhou

safety

arXiv Machine Learning

Jul 14

Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraints

arXiv:2509. 20114v3 Announce Type: replace Abstract: We study \emph{online episodic Constrained Markov Decision Processes} (CMDPs) under both stochastic and adversarial constraints.

By Francesco Emanuele Stradi, Eleonora Fidelia Chiefari, Matteo Castiglioni, Alberto Marchesi, Nicola Gatti

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 16

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding

arXiv:2602. 00781v2 Announce Type: replace Abstract: Online reinforcement learning in non-episodic, finite-horizon MDPs remains underexplored and is challenged by the need to estimate returns to a fixed terminal time.

By Jiamin Xu, Kyra Gan

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 2

Online Learning in MDPs with Partially Adversarial Transitions and Losses

arXiv:2602. 09474v2 Announce Type: replace Abstract: We study reinforcement learning in MDPs whose transition function is stochastic at most steps but may behave adversarially at a fixed subset of $\Lambda$ steps per episode.

By Ofir Schlisselberg, Tal Lancewicki, Yishay Mansour

reinforcement-learning safety

arXiv Machine Learning

Jun 16

Q-Learning with Fine-Grained Gap-Dependent Regret

arXiv:2510. 06647v2 Announce Type: replace-cross Abstract: We study fine-grained gap-dependent regret bounds for model-free reinforcement learning in episodic tabular Markov Decision Processes.

By Haochen Zhang, Zhong Zheng, Lingzhou Xue

reinforcement-learning

Related stories

Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions

Online Learning with Gradient-Variation Interval Regret

Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraints

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding

Online Learning in MDPs with Partially Adversarial Transitions and Losses

Q-Learning with Fine-Grained Gap-Dependent Regret