arXiv AI
Jun 3
Strongly Polynomial Time Complexity of Policy Iteration for $L_\infty$ Robust MDPs
arXiv:2601. 23229v2 Announce Type: replace Abstract: Markov decision processes (MDPs) are a fundamental model in sequential decision making.
Quantitative Analysis of $ω$-Regular Robust MDPs
Related stories
arXiv AI
Sep 30
Solving Robust POMDPs with Omega-regular Objectives via Partially Observable Stochastic Games
arXiv:2608.24986v2 Announce Type: replace Abstract: Robust POMDPs (RPOMDPs) generalize classical POMDPs to the setting where exact transition probabilities are not known -- rather, they are only know...
Hugging Face Trending Papers
Aug 3
Optimizing Minimax Regret in Uncertain MDPs with Small Sets of Policies
Sequential decision-making in real-world applications often involves uncertainty about the environment's model. Uncertain Markov decision processes (UMDPs) represent the possible environments as a set of MDPs with shared states and actions but potentially different transition probabilities and rewards.
arXiv Machine Learning
Aug 4
Optimizing Minimax Regret in Uncertain MDPs with Small Sets of Policies
arXiv:2608. 02509v1 Announce Type: cross Abstract: Sequential decision-making in real-world applications often involves uncertainty about the environment's model.
arXiv Machine Learning
Oct 3
Linear Programming Representations and Strongly Polynomial Algorithms for Robust Markov Decision Processes
The paper presents linear programming formulations and strongly polynomial algorithms for robust Markov decision processes (RMDPs) with rational polyhedral state-action rectangular uncertainty in rewards and transitions. By encoding a finite sequence of robust policy-iteration steps, a single LP is constructed whose optimal solutions recover the robust optimal value and all optimal stationary randomized policies. The authors provide a general complexity analysis of robust policy iteration, improving known bounds for α1 and α1∞ RMDPs and establishing new strongly polynomial bounds for general interval, weighted α1, and Wasserstein RMDPs, as well as turn‑based stochastic games with these uncertainty sets.
arXiv AI
Jul 14
Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning
arXiv:2506. 07040v4 Announce Type: replace-cross Abstract: We study model-free methods for distributionally robust infinite-horizon average-reward Markov decision processes (MDPs).
Hugging Face Trending Papers
Sep 3
Robust PAC Learning of Concurrent Stochastic Games
We introduce the first Probably Approximately Correct (PAC) learning framework for general-sum concurrent stochastic games (CSGs) with transition uncertainty, while addressing the challenge of Nash equilibrium (NE) existence. Our algorithm maintains data-driven $L^1$ confidence sets over transition kernels and solves a robust CSG to compute a social-welfare optimal $\varepsilon$-NE, using a robust MDP-based exploration mechanism to drive joint state-action coverage.
arXiv AI
Oct 15
Certifying Almost-Sure Reachability in Unknown MDPs
arXiv:2610.18119v1 Announce Type: cross Abstract: {\em Specification-Guided Reinforcement Learning (RL)} has largely concerned itself with \emph{learning} an optimal policy against a formal specifica...
arXiv Machine Learning
Aug 4
Near-Optimal Reinforcement Learning for Constrained Recurrence Objectives
arXiv:2511. 19849v2 Announce Type: replace-cross Abstract: Recurrence objectives, where a target region must be visited infinitely often, are a fundamental class of specifications for Markov decision processes (MDPs) and form the core of $\omega$-regular and linear temporal logic (LTL) objectives.
arXiv Machine Learning
Sep 4
Robust PAC Learning of Concurrent Stochastic Games
The paper presents the first PAC learning framework for general-sum concurrent stochastic games with uncertain transitions, addressing the challenge of Nash equilibrium existence. It introduces data‑driven L¹ confidence sets over transition kernels and a robust CSG solver that computes a social‑welfare optimal ε‑NE, or provides a certificate that no exact NE exists. The algorithm achieves polynomial sample complexity under a minimum reachability condition and is validated on benchmark CSGs with near‑optimal performance.
arXiv AI
Aug 19
Adaptive Policy Portfolios for Robust Markov Decision Processes
The paper introduces adaptive policy portfolios for robust Markov decision processes, where a finite set of memoryless randomized policies is synthesized offline and paired with an online selector. It defines robust regret as a measure of portfolio quality, comparing each portfolio member to the optimal policy for each plausible environment. The authors provide a complexity-theoretic analysis of portfolio certification and synthesis, showing that even deterministic portfolios in simple settings are highly complex, and present an offline construction method that can be specialized at runtime.
arXiv Machine Learning
Oct 5
On the Convergence of Success Conditioning for Policy Optimization
arXiv:2610.03642v1 Announce Type: new Abstract: Success conditioning is a strategy for improving decision-making policies in stochastic environments; it updates a policy by increasing the probability...