arXiv Machine Learning October 9, 2026 By Mingyi Li, Taira Tsuchiya

Closing the Horizon Gap in Policy Optimization for Adversarial MDPs

Read the original on arXiv Machine Learning →

The paper addresses policy optimization in online episodic tabular MDPs with adversarial losses and bandit feedback. By employing regularized Q‑functions, the authors eliminate the extra horizon factor present in previous policy‑optimization regret bounds, achieving high‑probability regret of ≥O(√{HS(H+A)T}) for known transitions and ≥O(HS√{AT}) for unknown transitions. These results improve horizon dependence and match the best-known bounds, with extensions to adversarial linear‑mixture MDPs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 2

Online Learning in MDPs with Partially Adversarial Transitions and Losses

arXiv:2602. 09474v2 Announce Type: replace Abstract: We study reinforcement learning in MDPs whose transition function is stochastic at most steps but may behave adversarially at a fixed subset of $\Lambda$ steps per episode.

By Ofir Schlisselberg, Tal Lancewicki, Yishay Mansour

reinforcement-learning safety

arXiv Machine Learning

Jun 3

Data- and Variance-dependent Regret Bounds for Online Tabular MDPs

arXiv:2602. 01903v2 Announce Type: replace Abstract: This work studies online episodic tabular Markov decision processes (MDPs) with known transitions and develops best-of-both-worlds algorithms that achieve refined data-dependent regret bounds in the adversarial regime and variance-dependent regret bounds in the stochastic regime.

By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

reinforcement-learning safety

arXiv Machine Learning

Jul 1

Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions

arXiv:2606. 31769v1 Announce Type: new Abstract: We study policy optimization for online episodic tabular Markov decision processes with unknown transition kernels, aiming for best-of-both-worlds guarantees together with data-dependent regret bounds.

By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

reinforcement-learning safety

arXiv Machine Learning

Oct 9

Toward Optimal Regret in Adversarial MDPs with Stochastic Hard Constraints

arXiv:2610. 12153v1 Announce Type: new Abstract: We study episodic constrained Markov decision processes with adversarial losses under stochastic hard constraints.

By Qian Zuo, Francesco Emanuele Stradi

reinforcement-learning safety

arXiv Machine Learning

Sep 21

Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards

arXiv:2603.27884v2 Announce Type: replace Abstract: We study safe reinforcement learning in finite-horizon linear mixture constrained Markov decision processes (CMDPs) with adversarial rewards under...

By Kihyun Yu, Seoungbin Bae, Dabeen Lee

reinforcement-learning safety

arXiv Machine Learning

Jun 2

Minimax-Optimal Policy Regret in Partially Observable Markov Games

arXiv:2606. 02363v1 Announce Type: new Abstract: We study sequential decision-making in partially observable environments against strategic, adaptive opponents, modeled as partially observable Markov games (POMGs).

By Raman Arora

One email a morning, machine-written

Related stories

Online Learning in MDPs with Partially Adversarial Transitions and Losses

Data- and Variance-dependent Regret Bounds for Online Tabular MDPs

Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions

Toward Optimal Regret in Adversarial MDPs with Stochastic Hard Constraints

Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards

Minimax-Optimal Policy Regret in Partially Observable Markov Games