arXiv Machine Learning June 3, 2026 By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

Data- and Variance-dependent Regret Bounds for Online Tabular MDPs

Read the original on arXiv Machine Learning →

arXiv:2602. 01903v2 Announce Type: replace Abstract: This work studies online episodic tabular Markov decision processes (MDPs) with known transitions and develops best-of-both-worlds algorithms that achieve refined data-dependent regret bounds in the adversarial regime and variance-dependent regret bounds in the stochastic regime.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 1

Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions

arXiv:2606. 31769v1 Announce Type: new Abstract: We study policy optimization for online episodic tabular Markov decision processes with unknown transition kernels, aiming for best-of-both-worlds guarantees together with data-dependent regret bounds.

By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

reinforcement-learning safety

arXiv Machine Learning

Jun 3

Online Learning with Gradient-Variation Interval Regret

arXiv:2606. 03831v1 Announce Type: new Abstract: This paper investigates non-stationary online learning using the metric of interval regret, which requires an online algorithm to perform well over every time interval.

By Yan-Feng Xie, Shuche Wang, Peng Zhao, Zhi-Hua Zhou

safety

arXiv Machine Learning

Jul 14

Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraints

arXiv:2509. 20114v3 Announce Type: replace Abstract: We study \emph{online episodic Constrained Markov Decision Processes} (CMDPs) under both stochastic and adversarial constraints.

By Francesco Emanuele Stradi, Eleonora Fidelia Chiefari, Matteo Castiglioni, Alberto Marchesi, Nicola Gatti

reinforcement-learning benchmarks safety

arXiv Machine Learning

Aug 27

Beyond Optimal Rates in Stochastic Optimization: Trajectory-Adaptive Stopping Rules

arXiv:2608. 25551v1 Announce Type: new Abstract: Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory.

By Liviu Aolaritei, Lucas L\'evy, Francis Bach, Michael I. Jordan

arXiv Machine Learning

1d ago

Rate-Optimal Algorithm for Adversarial Linear CMDPs

The paper introduces a new primal–dual algorithm for episodic adversarial linear constrained Markov decision processes (CMDPs) with unknown transitions. It achieves a rate‑optimal ×O(√K) regret and cumulative constraint violation, improving upon the previous ×O(K^{3/4}) bound and eliminating the need for Slater’s condition. The method combines adaptive FTRL, contracted value estimation, and an exponential Lyapunov function, enabling uniform concentration over the value function class and computational efficiency independent of the state‑space size.

By Kihyun Yu, Honghao Wei, Dabeen Lee

reinforcement-learning safety

arXiv Machine Learning

Jun 16

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding

arXiv:2602. 00781v2 Announce Type: replace Abstract: Online reinforcement learning in non-episodic, finite-horizon MDPs remains underexplored and is challenged by the need to estimate returns to a fixed terminal time.

By Jiamin Xu, Kyra Gan

reinforcement-learning benchmarks

One email a morning, machine-written

Related stories

Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions

Online Learning with Gradient-Variation Interval Regret

Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraints

Beyond Optimal Rates in Stochastic Optimization: Trajectory-Adaptive Stopping Rules

Rate-Optimal Algorithm for Adversarial Linear CMDPs

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding