The monograph "Statistical Foundations of Reinforcement Learning: A Non-Asymptotic Perspective" surveys recent advances in finite-sample statistical theory for reinforcement learning. It focuses on tabular Markov decision processes and examines three sampling settings—generative model, online, and offline RL—alongside model-based and model-free algorithmic paradigms. The work emphasizes minimax-optimal sample complexity, provides self-contained analyses, and briefly explores extensions such as robust, multi-agent, federated, and human-feedback RL.
arXiv:2602. 12107v2 Announce Type: replace-cross Abstract: We study offline reinforcement learning under $Q^\star$-approximation and partial coverage, a setting that motivates practical algorithms such as Conservative $Q$-Learning (CQL; Kumar et al.
arXiv:2607. 17201v1 Announce Type: cross Abstract: In this work we study the Best Policy Identification (BPI) problem in online, tabular Reinforcement Learning.
arXiv:2512. 06244v2 Announce Type: replace Abstract: The exploration-exploitation dilemma in reinforcement learning (RL) is a fundamental challenge to efficient RL algorithms.
arXiv:2512. 14617v2 Announce Type: replace-cross Abstract: Many practical decision-making problems involve tasks whose success depends on the entire system history, rather than on achieving a state with desired properties.
arXiv:2602. 00781v2 Announce Type: replace Abstract: Online reinforcement learning in non-episodic, finite-horizon MDPs remains underexplored and is challenged by the need to estimate returns to a fixed terminal time.
The paper introduces a sample‑efficient asynchronous Q‑learning algorithm for robust reinforcement learning in infinite‑horizon, discounted robust MDPs with total‑variation uncertainty. It proves that, given a single Markovian trajectory, the algorithm achieves an entrywise ε‑approximation of the optimal Q‑function with a sample complexity bounded by a nearly minimax‑optimal expression involving the discount factor, uncertainty level, mixing time, and occupancy probability. This result improves significantly over prior bounds for robust RL.
arXiv:2606. 14095v1 Announce Type: new Abstract: We study the sample complexity of learning in average-reward weakly-coupled Markov decision processes (WCMDPs) and Restless Bandits (RBs) under a generative model.
arXiv:2608. 06545v1 Announce Type: new Abstract: Distributionally robust Markov decision processes provide a principled framework for sequential decision making under model uncertainty.
arXiv:2510. 19528v2 Announce Type: replace-cross Abstract: We investigate the fundamental problem of leveraging offline data to accelerate online reinforcement learning - a direction with strong potential but limited theoretical grounding.
The paper introduces state abstractions that preserve the difference of Q‑functions for offline reinforcement learning, aiming to exclude irrelevant dynamics from rich state data. It proposes a dynamic generalization of the R‑learner that uses orthogonal estimation and sparse learning to estimate the Q‑function contrast, achieving faster convergence and consistency under a margin condition. Experiments on simulated and simulator‑augmented real data show variance reductions and demonstrate that the necessary information for sequential decision‑making can be smaller than that required for full state prediction.
arXiv:2606. 29980v1 Announce Type: new Abstract: Zero-shot Transfer in Reinforcement Learning (RL) aims to train an agent that can generate optimal policies for any reward function, without additional learning at transfer time, while training only on reward-free trajectories.
arXiv:2606. 04182v1 Announce Type: cross Abstract: We formulate the problem of \emph{exact unlearning} in reinforcement learning, where the goal is to design an efficient framework that enables the removal of any user's data upon deletion request, i.