arXiv Machine Learning October 3, 2026 By Kihyun Yu, Honghao Wei, Dabeen Lee

Rate-Optimal Algorithm for Adversarial Linear CMDPs

Read the original on arXiv Machine Learning →

The paper introduces a new primal–dual algorithm for episodic adversarial linear constrained Markov decision processes (CMDPs) with unknown transitions. It achieves a rate‑optimal ×O(√K) regret and cumulative constraint violation, improving upon the previous ×O(K^{3/4}) bound and eliminating the need for Slater’s condition. The method combines adaptive FTRL, contracted value estimation, and an exponential Lyapunov function, enabling uniform concentration over the value function class and computational efficiency independent of the state‑space size.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Sep 21

Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards

arXiv:2603.27884v2 Announce Type: replace Abstract: We study safe reinforcement learning in finite-horizon linear mixture constrained Markov decision processes (CMDPs) with adversarial rewards under...

By Kihyun Yu, Seoungbin Bae, Dabeen Lee

reinforcement-learning safety

arXiv Machine Learning

Jul 14

Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraints

arXiv:2509. 20114v3 Announce Type: replace Abstract: We study \emph{online episodic Constrained Markov Decision Processes} (CMDPs) under both stochastic and adversarial constraints.

By Francesco Emanuele Stradi, Eleonora Fidelia Chiefari, Matteo Castiglioni, Alberto Marchesi, Nicola Gatti

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jul 1

Constrained Online Convex Optimization without Slater's Condition

arXiv:2606. 31480v1 Announce Type: new Abstract: We study constrained online convex optimization with adversarial losses and stochastic or adversarial constraints.

By Kihyun Yu, Junehee Lee, Dabeen Lee

safety

arXiv Machine Learning

Jun 25

Bias-Controlled Primal-Dual Natural Actor-Critic: Optimal Rates for Constrained Multi-Objective Average-Reward RL

arXiv:2606. 25012v1 Announce Type: new Abstract: Many reinforcement learning (RL) problems in the infinite-horizon average-reward setting require optimizing multiple conflicting objectives while satisfying multiple safety constraints.

By Ankur Naskar, Swetha Ganesh, Vaneet Aggarwal

agents reinforcement-learning safety

arXiv Machine Learning

Jul 31

Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs

arXiv:2607. 28390v1 Announce Type: new Abstract: Constrained Markov Decision Processes (CMDPs) provide a natural framework for reinforcement learning in safety-critical applications, where agents maximize long-term reward while satisfying long-term constraints.

By Ankur Naskar, Vaneet Aggarwal

agents reinforcement-learning safety

arXiv Machine Learning

Jun 3

Data- and Variance-dependent Regret Bounds for Online Tabular MDPs

arXiv:2602. 01903v2 Announce Type: replace Abstract: This work studies online episodic tabular Markov decision processes (MDPs) with known transitions and develops best-of-both-worlds algorithms that achieve refined data-dependent regret bounds in the adversarial regime and variance-dependent regret bounds in the stochastic regime.

By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

reinforcement-learning safety

One email a morning, machine-written

Related stories

Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards

Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraints

Constrained Online Convex Optimization without Slater's Condition

Bias-Controlled Primal-Dual Natural Actor-Critic: Optimal Rates for Constrained Multi-Objective Average-Reward RL

Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs

Data- and Variance-dependent Regret Bounds for Online Tabular MDPs