arXiv Machine Learning October 23, 2026 By Chenhao Zhou, Chao Zhang, Wensong Bai, Hanbin Zhao, Hui Qian

Primal-Dual Representation Learning for Low-Rank Constrained MDPs

Read the original on arXiv Machine Learning →

The paper introduces REP-PD-TEG, a representation‑learning algorithm for low‑rank continuous‑state constrained Markov Decision Processes (CMDPs) that achieves optimal ≥O(√K) regret in the soft cumulative‑violation setting while maintaining a ≥O(K^{3/4}) sample bound when exploratory actions are penalised. For the stricter hard cumulative‑violation scenario, REP-PD-H-TEG is proposed, which uses a bonus‑corrected lower‑confidence utility certificate and a stabilized dual search to guarantee both regret and hard‑violation bounds. Together, these methods provide the first theoretically guaranteed representation‑learning framework for low‑rank continuous‑state CMDPs with both soft and hard violation controls.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

reinforcement-learning

arXiv Machine Learning

Oct 3

Rate-Optimal Algorithm for Adversarial Linear CMDPs

The paper introduces a new primal–dual algorithm for episodic adversarial linear constrained Markov decision processes (CMDPs) with unknown transitions. It achieves a rate‑optimal ×O(√K) regret and cumulative constraint violation, improving upon the previous ×O(K^{3/4}) bound and eliminating the need for Slater’s condition. The method combines adaptive FTRL, contracted value estimation, and an exponential Lyapunov function, enabling uniform concentration over the value function class and computational efficiency independent of the state‑space size.

By Kihyun Yu, Honghao Wei, Dabeen Lee

reinforcement-learning safety

arXiv Machine Learning

Jun 9

Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy

arXiv:2606. 08779v1 Announce Type: new Abstract: Reinforcement Learning (RL) has emerged as a pivotal post-training paradigm, yet it frequently suffers from unpredictable sub-optimum performance or even training collapses.

By Jiashun Liu, Runze Liu, Xu Wan, Jing Liang, Hongyao Tang, Ling Pan

llms reinforcement-learning

arXiv Machine Learning

Oct 7

Decision-Focused Learning in MDPs: An Occupancy Measure Approach

arXiv:2610.08384v1 Announce Type: new Abstract: In this work, we consider decision-focused learning (DFL) for a Markov decision process (MDP), where existing methods differentiate through the KKT con...

By Zihao Zhao, Ashwath K. Karunakaram, Ali Eshragh, Yuexing Li, Kai Wang

reinforcement-learning

Hugging Face Trending Papers

Oct 6

Decision-Focused Learning in MDPs: An Occupancy Measure Approach

In this work, we consider decision-focused learning (DFL) for a Markov decision process (MDP), where existing methods differentiate through the KKT conditions of the Bellman equation and require solvi...

reinforcement-learning

arXiv Machine Learning

Sep 21

Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards

arXiv:2603.27884v2 Announce Type: replace Abstract: We study safe reinforcement learning in finite-horizon linear mixture constrained Markov decision processes (CMDPs) with adversarial rewards under...

By Kihyun Yu, Seoungbin Bae, Dabeen Lee

reinforcement-learning safety

arXiv Machine Learning

Aug 4

Near-Optimal Reinforcement Learning for Constrained Recurrence Objectives

arXiv:2511. 19849v2 Announce Type: replace-cross Abstract: Recurrence objectives, where a target region must be visited infinitely often, are a fundamental class of specifications for Markov decision processes (MDPs) and form the core of $\omega$-regular and linear temporal logic (LTL) objectives.

By Dominik Wagner, Leon Witzman, Luke Ong

reinforcement-learning safety

One email a morning, machine-written

Related stories

Rate-Optimal Algorithm for Adversarial Linear CMDPs

Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy

Decision-Focused Learning in MDPs: An Occupancy Measure Approach

Decision-Focused Learning in MDPs: An Occupancy Measure Approach

Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards

Near-Optimal Reinforcement Learning for Constrained Recurrence Objectives