The paper introduces adaptive policy portfolios for robust Markov decision processes, where a finite set of memoryless randomized policies is synthesized offline and paired with an online selector. It defines robust regret as a measure of portfolio quality, comparing each portfolio member to the optimal policy for each plausible environment. The authors provide a complexity-theoretic analysis of portfolio certification and synthesis, showing that even deterministic portfolios in simple settings are highly complex, and present an offline construction method that can be specialized at runtime.
By Kasper Engelen, Sebastian Junges, Guillermo A. P\'{e}rez, Marnix Suilen
arXiv:2606. 08791v1 Announce Type: cross Abstract: We study the problem of auditing a black-box algorithmic decision-maker from observable inputs and outputs alone.
By Irene Aldridge
The paper presents linear programming formulations and strongly polynomial algorithms for robust Markov decision processes (RMDPs) with rational polyhedral state-action rectangular uncertainty in rewards and transitions. By encoding a finite sequence of robust policy-iteration steps, a single LP is constructed whose optimal solutions recover the robust optimal value and all optimal stationary randomized policies. The authors provide a general complexity analysis of robust policy iteration, improving known bounds for α1 and α1∞ RMDPs and establishing new strongly polynomial bounds for general interval, weighted α1, and Wasserstein RMDPs, as well as turn‑based stochastic games with these uncertainty sets.
By Han Zhong, Yinyu Ye
arXiv:2601. 23229v2 Announce Type: replace Abstract: Markov decision processes (MDPs) are a fundamental model in sequential decision making.
By Ali Asadi, Krishnendu Chatterjee, Ehsan Goharshady, Mehrdad Karrabi, Alipasha Montaseri, Carlo Pagano
Robust Markov Decision Processes (RMDPs) generalize classical MDPs by allowing uncertainty in transition probabilities and optimizing against their worst-case realization. We consider $(s,a)$-rectangu...
arXiv:2609. 14959v1 Announce Type: new Abstract: We study decentralized learning of Nash equilibria (NE) in infinite-horizon discounted Markov games under bandit feedback, focusing on Markov $\alpha$-potential games.
By S. Rasoul Etesami
arXiv:2608. 02509v1 Announce Type: cross Abstract: Sequential decision-making in real-world applications often involves uncertainty about the environment's model.
By Sterre Lutz, Dani\"el Vos, Matthijs T. J. Spaan, Anna Lukina
arXiv:2511. 19849v2 Announce Type: replace-cross Abstract: Recurrence objectives, where a target region must be visited infinitely often, are a fundamental class of specifications for Markov decision processes (MDPs) and form the core of $\omega$-regular and linear temporal logic (LTL) objectives.
By Dominik Wagner, Leon Witzman, Luke Ong
Sequential decision-making in real-world applications often involves uncertainty about the environment's model. Uncertain Markov decision processes (UMDPs) represent the possible environments as a set of MDPs with shared states and actions but potentially different transition probabilities and rewards.
arXiv:2511.08097v2 Announce Type: replace-cross
Abstract: We consider a general infinite horizon Heterogeneous Restless multi-armed Bandit (RMAB). Heterogeneity is a fundamental problem for many real...
By Dheeraj Narasimha, Nicolas Gast
arXiv:2606. 27448v1 Announce Type: new Abstract: This paper studies the problem of regret minimization in Markovian bandits with \emph{non-observable states} and possibly \emph{constrained} decision epochs.
By Thomas Hira, Victor Boone, Urtzi Ayesta, Ina Maria Verloop
arXiv:2512. 09850v2 Announce Type: replace Abstract: We introduce Conformal Bandits, a novel framework integrating Conformal Prediction (CP) into bandit problems, a classic paradigm for sequential decision-making under uncertainty.
By Simone Cuonzo, Nina Deliu