The paper introduces a new primal–dual algorithm for episodic adversarial linear constrained Markov decision processes (CMDPs) with unknown transitions. It achieves a rate‑optimal ×O(√K) regret and cumulative constraint violation, improving upon the previous ×O(K^{3/4}) bound and eliminating the need for Slater’s condition. The method combines adaptive FTRL, contracted value estimation, and an exponential Lyapunov function, enabling uniform concentration over the value function class and computational efficiency independent of the state‑space size.
By Kihyun Yu, Honghao Wei, Dabeen Lee
arXiv:2606. 31480v1 Announce Type: new Abstract: We study constrained online convex optimization with adversarial losses and stochastic or adversarial constraints.
By Kihyun Yu, Junehee Lee, Dabeen Lee
arXiv:2602. 01903v2 Announce Type: replace Abstract: This work studies online episodic tabular Markov decision processes (MDPs) with known transitions and develops best-of-both-worlds algorithms that achieve refined data-dependent regret bounds in the adversarial regime and variance-dependent regret bounds in the stochastic regime.
By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi
arXiv:2509. 16586v2 Announce Type: replace Abstract: Recent advances have significantly improved our understanding of the sample complexity of learning in average-reward Markov decision processes (AMDPs) under the generative model.
By Yukuan Wei, Xudong Li, Lin F. Yang
arXiv:2602. 09474v2 Announce Type: replace Abstract: We study reinforcement learning in MDPs whose transition function is stochastic at most steps but may behave adversarially at a fixed subset of $\Lambda$ steps per episode.
By Ofir Schlisselberg, Tal Lancewicki, Yishay Mansour
arXiv:2606. 08028v1 Announce Type: new Abstract: We study high-probability regret bounds for online convex optimization (OCO) with strongly convex losses and establish three results that resolve open questions at the intersection of noise adaptivity, feedback structure, and constraint satisfaction.
By Wentao Zhang, Yutong Zhang, Wentao Mo
The paper introduces a new approach to safety in contextual bandits with continuous actions, focusing on high‑probability constraints on the realized cost rather than expected cost. It presents the High‑Probability Constrained UCB algorithm, which balances reward exploration with conservative safety estimation, and provides theoretical regret guarantees for linear models and extensions to general function classes. Experiments demonstrate that this realized‑cost safety framework significantly reduces safety violations compared to expected‑cost constrained methods.
By Spyros Dragazis, Aldo Pacchiano
arXiv:2609. 06921v1 Announce Type: cross Abstract: We study constrained online convex optimization with adversarial constraints when constraint values and gradients are observed through unbiased noise.
By Vaneet Aggarwal
arXiv:2607. 29460v1 Announce Type: new Abstract: Heavy-tailed distributions arise naturally in sequential decision-making problems such as financial investment, online advertising, and network management, where rare but extreme outcomes can dominate performance.
By Gianmarco Genalti, Alberto Maria Metelli
arXiv:2605.15692v2 Announce Type: replace-cross
Abstract: We study episodic reinforcement learning with fixed reward and transition functions, but with episode-dependent admissible action sets that a...
By Zijun Chen, Zihan Zhang
arXiv:2502. 16744v3 Announce Type: replace Abstract: In adversarial Constrained Online Convex Optimization (COCO), a learner selects actions from a fixed convex set while seeking both low regret and low cumulative constraint violation (CCV) under time-varying constraints.
By Yiyang Lu, Mohammad Pedramfar, Mengbo Wang, Vaneet Aggarwal
arXiv:2603.27884v2 Announce Type: replace
Abstract: We study safe reinforcement learning in finite-horizon linear mixture constrained Markov decision processes (CMDPs) with adversarial rewards under...
By Kihyun Yu, Seoungbin Bae, Dabeen Lee