arXiv Machine Learning
Oct 23

Primal-Dual Representation Learning for Low-Rank Constrained MDPs

The paper introduces REP-PD-TEG, a representation‑learning algorithm for low‑rank continuous‑state constrained Markov Decision Processes (CMDPs) that achieves optimal ≥O(√K) regret in the soft cumulative‑violation setting while maintaining a ≥O(K^{3/4}) sample bound when exploratory actions are penalised. For the stricter hard cumulative‑violation scenario, REP-PD-H-TEG is proposed, which uses a bonus‑corrected lower‑confidence utility certificate and a stabilized dual search to guarantee both regret and hard‑violation bounds. Together, these methods provide the first theoretically guaranteed representation‑learning framework for low‑rank continuous‑state CMDPs with both soft and hard violation controls.

By Chenhao Zhou, Chao Zhang, Wensong Bai, Hanbin Zhao, Hui Qian