The paper establishes a game-theoretic framework for bisimulation metrics in undiscounted reinforcement learning settings, including total- and average-reward problems. By interpreting metric operators as Bellman–Shapley operators of zero-sum turn-based games, it defines unique metrics even when the discount factor equals one. The authors also explore how metric finiteness relates to MDP structure, introduce a flexible action-space modification framework, and provide bounds linking these metrics to optimal value differences and aggregation-based planning guarantees.
arXiv:2511. 19849v2 Announce Type: replace-cross Abstract: Recurrence objectives, where a target region must be visited infinitely often, are a fundamental class of specifications for Markov decision processes (MDPs) and form the core of $\omega$-regular and linear temporal logic (LTL) objectives.
The paper establishes formal theorems linking game theory and reinforcement learning, showing that a game can be viewed as a Markov decision process in three distinct ways—definitional identity, exact reduction, or structural analogy. It systematically maps every standard game class and solution concept to corresponding reinforcement learning models, training procedures, and complexity results. The work also identifies where these correspondences break down, such as non‑stationarity, impossibility of uncoupled convergence to Nash equilibrium, and gaps between coarse correlated and correlated equilibria.
arXiv:2608. 09389v1 Announce Type: cross Abstract: This note aims to serve as an entry point to the literature on learning in games, a topic with significant theoretical appeal and a wide range of applications -- from machine learning and data science to economics and beyond.
arXiv:2610.14988v1 Announce Type: cross Abstract: The average-reward setting for reinforcement learning is challenging because its Bellman operator is known to contract in span only when the underlyi...
arXiv:2606. 16729v1 Announce Type: new Abstract: While there is an extensive body of work characterizing the sample complexity of discounted cumulative-reward MDPs, finite sample analyses for average-reward MDPs have been limited, and most existing works rely on restrictive assumptions such as ergodicity or access to a generative model.
arXiv:2606. 04053v1 Announce Type: cross Abstract: The Boolean Task Algebra (BTA) provides a principled framework for zero-shot task composition in reinforcement learning by equipping goal-reaching tasks with Boolean operations.
Robust Markov Decision Processes (RMDPs) generalize classical MDPs by allowing uncertainty in transition probabilities and optimizing against their worst-case realization. We consider $(s,a)$-rectangu...
We study infinite-horizon average-reward constrained Markov decision processes (CMDPs) under the weakly communicating assumption. Existing high-probability guarantees for this setting either require c...
arXiv:2610. 08745v1 Announce Type: new Abstract: We study linear bandits under exact sliding-window constraints, where every consecutive block of actions must belong to a prescribed feasible set.
arXiv:2610.03621v1 Announce Type: cross Abstract: There has been a surge of recent work on correlated equilibrium concepts in Markov games. However, existing results focus on concepts weaker than nor...
arXiv:2506. 13862v2 Announce Type: replace-cross Abstract: In Reinforcement Learning (RL), regularization with a Kullback-Leibler divergence that penalizes large deviations between successive policies has emerged as a popular tool both in theory and practice.
The paper introduces a framework for reinforcement learning in Markov decision processes where policies are ordered by the information they provide, allowing data from one policy to estimate the performance of others. An epoch‑based algorithm is developed for infinite‑horizon average‑cost MDPs with continuous state and action spaces, achieving regret that scales with the width of the policy class’s information order. The authors apply the framework to lost‑sales inventory, dual‑sourcing, and state‑dependent queueing systems, demonstrating improved regret bounds and empirical performance without requiring global convexity or specific arrival‑rate structures.