arXiv Machine Learning
Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
1d ago
Rate-Optimal Algorithm for Adversarial Linear CMDPs
The paper introduces a new primal–dual algorithm for episodic adversarial linear constrained Markov decision processes (CMDPs) with unknown transitions. It achieves a rate‑optimal ×O(√K) regret and cumulative constraint violation, improving upon the previous ×O(K^{3/4}) bound and eliminating the need for Slater’s condition. The method combines adaptive FTRL, contracted value estimation, and an exponential Lyapunov function, enabling uniform concentration over the value function class and computational efficiency independent of the state‑space size.
arXiv Machine Learning
Jul 31
Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs
arXiv:2607. 28390v1 Announce Type: new Abstract: Constrained Markov Decision Processes (CMDPs) provide a natural framework for reinforcement learning in safety-critical applications, where agents maximize long-term reward while satisfying long-term constraints.
arXiv Machine Learning
Sep 14
A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning
The paper presents a unified framework for regularization-based robust reinforcement learning by deriving upper bounds on the performance gap between nominal and worst-case policies. These bounds are expressed as a regularization objective plus a KL-divergence penalty, explaining why KL penalties enhance robustness. The authors reformulate robust training as a constrained optimization problem, updating the Lagrange multiplier jointly with the policy to automatically tune regularization, and validate the approach with extensive adversarial evaluations on continuous control tasks.
arXiv Machine Learning
Jul 14
Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraints
arXiv:2509. 20114v3 Announce Type: replace Abstract: We study \emph{online episodic Constrained Markov Decision Processes} (CMDPs) under both stochastic and adversarial constraints.
arXiv Machine Learning
Jul 1
Constrained Online Convex Optimization without Slater's Condition
arXiv:2606. 31480v1 Announce Type: new Abstract: We study constrained online convex optimization with adversarial losses and stochastic or adversarial constraints.
arXiv Machine Learning
Jun 3
Data- and Variance-dependent Regret Bounds for Online Tabular MDPs
arXiv:2602. 01903v2 Announce Type: replace Abstract: This work studies online episodic tabular Markov decision processes (MDPs) with known transitions and develops best-of-both-worlds algorithms that achieve refined data-dependent regret bounds in the adversarial regime and variance-dependent regret bounds in the stochastic regime.