The paper introduces REP-PD-TEG, a representation‑learning algorithm for low‑rank continuous‑state constrained Markov Decision Processes (CMDPs) that achieves optimal ≥O(√K) regret in the soft cumulative‑violation setting while maintaining a ≥O(K^{3/4}) sample bound when exploratory actions are penalised. For the stricter hard cumulative‑violation scenario, REP-PD-H-TEG is proposed, which uses a bonus‑corrected lower‑confidence utility certificate and a stabilized dual search to guarantee both regret and hard‑violation bounds. Together, these methods provide the first theoretically guaranteed representation‑learning framework for low‑rank continuous‑state CMDPs with both soft and hard violation controls.
By Chenhao Zhou, Chao Zhang, Wensong Bai, Hanbin Zhao, Hui Qian
The paper introduces a new primal–dual algorithm for episodic adversarial linear constrained Markov decision processes (CMDPs) with unknown transitions. It achieves a rate‑optimal ×O(√K) regret and cumulative constraint violation, improving upon the previous ×O(K^{3/4}) bound and eliminating the need for Slater’s condition. The method combines adaptive FTRL, contracted value estimation, and an exponential Lyapunov function, enabling uniform concentration over the value function class and computational efficiency independent of the state‑space size.
By Kihyun Yu, Honghao Wei, Dabeen Lee
We study infinite-horizon average-reward constrained Markov decision processes (CMDPs) under the weakly communicating assumption. Existing high-probability guarantees for this setting either require c...
The paper introduces a computationally efficient algorithm for infinite-horizon average-reward constrained Markov decision processes (CMDPs) under weak communication. It achieves a high-probability regret and cumulative constraint violation of ×O(√T) in the tabular setting, matching optimal dependence up to logarithmic factors. The method augments the state with cumulative constraint violation, reshapes rewards using a Huber potential, and applies finite-horizon approximation with optimistic value iteration to maintain bounded per-step rewards.
By Kihyun Yu, Seoungbin Bae, Dabeen Lee
arXiv:2603.27884v2 Announce Type: replace
Abstract: We study safe reinforcement learning in finite-horizon linear mixture constrained Markov decision processes (CMDPs) with adversarial rewards under...
By Kihyun Yu, Seoungbin Bae, Dabeen Lee
arXiv:2602. 03778v2 Announce Type: replace-cross Abstract: Tail-end risk measures such as static conditional value-at-risk (CVaR) are used in safety-critical applications to prevent rare, yet catastrophic events.
By Aneri Muni, Vincent Taboga, Esther Derman, Pierre-Luc Bacon, Erick Delage
arXiv:2610.08384v1 Announce Type: new
Abstract: In this work, we consider decision-focused learning (DFL) for a Markov decision process (MDP), where existing methods differentiate through the KKT con...
By Zihao Zhao, Ashwath K. Karunakaram, Ali Eshragh, Yuexing Li, Kai Wang
arXiv:2604. 00531v2 Announce Type: replace Abstract: Multi-task representation learning exploits the shared structure among related tasks by learning a common latent representation, thereby improving sample efficiency.
By Jiabin Lin, Shana Moothedath
In this work, we consider decision-focused learning (DFL) for a Markov decision process (MDP), where existing methods differentiate through the KKT conditions of the Bellman equation and require solvi...
The paper challenges the common assumption that the successor measure in reinforcement learning is approximately low-rank, showing instead that a low-rank structure emerges in a shifted successor measure that ignores initial transitions. It provides finite-sample guarantees for estimating this low-rank approximation, introduces Type II Poincaré inequalities to bound spectral recoverability, and links the necessary shift to the decay of high-order singular values and local mixing properties. Experiments confirm that shifting the successor measure improves goal-conditioned RL performance.
By Bastien Dubail, Stefan Stojanovic, Alexandre Prouti\`ere
arXiv:2607. 19854v1 Announce Type: new Abstract: We study horizon-free regret minimization for finite-horizon time-homogeneous tabular Markov decision processes with $S$ states, $A$ actions, horizon $H$, and per-trajectory total reward bounded by $1$.
By Runlong Zhou, Zihan Zhang, Maryam Fazel, Simon S. Du
arXiv:2510. 06647v2 Announce Type: replace-cross Abstract: We study fine-grained gap-dependent regret bounds for model-free reinforcement learning in episodic tabular Markov Decision Processes.
By Haochen Zhang, Zhong Zheng, Lingzhou Xue