The paper investigates regression with bounded responses, comparing two learning frameworks: model selection aggregation, which requires improper algorithms to achieve minimax excess risk, and universal learning, where empirical risk minimization suffices for exponential learning rates. For finite hypothesis classes, the authors show that the $Q$-aggregation estimator simultaneously attains minimax optimal tails and exponential universal rates, while other common estimators fail to do so. For countably infinite classes, they prove an inherent trade‑off between exponential universal and minimax uniform rates, resolved by combining optimal algorithms from each framework via $Q$-aggregation.
By Mikael M{\o}ller H{\o}gsgaard, Patrick Rebeschini, Tobias Wegel
arXiv:2609.09981v1 Announce Type: new
Abstract: We study offline inference for the optimal value in reinforcement learning. Two new nuisances are derived as fixed points of a self-induced Bellman equ...
By Nan Lu, Ethan Lee, James M. Robins, David Simchi-Levi, Junwei Lu
arXiv:2511. 22226v2 Announce Type: replace Abstract: The standard theory of model-free reinforcement learning assumes that the environment dynamics are stationary and that agents are decoupled from their environment, such that policies are treated as being separate from the world they inhabit.
By Alexander Meulemans, Rajai Nasser, Maciej Wo{\l}czyk, Marissa A. Weis, Seijin Kobayashi, Blake Richards, Guillaume Lajoie, Angelika Steger, Marcus Hutter, James Manyika, Rif A. Saurous, Jo\~ao Sacramento, Blaise Ag\"uera y Arcas
arXiv:2607. 17823v1 Announce Type: new Abstract: Reinforcement Learning is a cornerstone technique for modern large reasoning models.
By Riccardo Poiani, Martino Bernasconi, Andrea Celli
arXiv:2605. 23146v3 Announce Type: replace-cross Abstract: Classical reinforcement learning assumes the agent interacts with a fixed environment whose behavior does not depend on the agent's policy.
By Manish Aryal, Faiyaz Azam, Agnivo Banerjee, Syed Mahir Ahamed, Sai Sidhanth Manoharan Jayanthi, Allegra Laro, Cl\'ement Legentilhomme, Andrew Lin, Florian Lorkowski, Marina P\'erez del Valle, Radman Rakhshandehroo, Patric Rommel, Emanuel Ruzak, Nathan Theng, Paul Yushin Rapoport
The paper investigates how to infer agents’ underlying incentives in strategic multi‑agent settings, where traditional inverse reinforcement learning (IRL) may fail to produce rewards that guide optimal play. It introduces the concept of strategic identifiability and shows that learning from a fixed pair of policies can be insufficient regardless of sample size. To address this, the authors propose an active IRL algorithm for two‑player repeated Stackelberg games, combining maximum‑likelihood estimation with experimental design to achieve finite‑sample guarantees on sub‑optimality, and validate the approach in an anti‑poaching scenario.
By Antoine Bergerault, Cyrille Kone, Giorgia Ramponi
arXiv:2602. 12963v2 Announce Type: replace Abstract: An important question in the field of AI is the extent to which successful behaviour requires an internal representation of the world.
By Alfred Harwood, Jose Faustino, Alex Altair
V‑SHARE is an advantage‑based value‑learning algorithm that blends cross‑action value sharing with an importance‑weighted Bellman‑residual correction. The paper presents the first full theoretical analysis of such an algorithm, offering both asymptotic convergence and finite‑sample guarantees. It also introduces schedules that balance deterministic contraction against importance‑weighting noise, and a two‑timescale analysis that eliminates a residual‑weight threshold while maintaining convergence.
By Kintan Saha, Hamed Masoudi, Xinyi Sheng, Dominik Baumann, Joni Pajarinen, Arsenii Mustafin
The paper introduces Robust Fed-Q, a federated Q‑learning algorithm designed for settings where multiple agents interact with a shared Markov Decision Process and communicate through a central server. It combines model‑based and model‑free reinforcement learning techniques with a median‑of‑means strategy from robust statistics to handle a small fraction of adversarial agents. The authors prove that Robust Fed-Q achieves exact convergence to the optimal value function with high probability, attains near‑optimal finite‑time rates that benefit from collaboration, and requires only “~O(1)” communication rounds per guarantee.
By Sreejeet Maity, Aritra Mitra
arXiv:2506. 13862v2 Announce Type: replace-cross Abstract: In Reinforcement Learning (RL), regularization with a Kullback-Leibler divergence that penalizes large deviations between successive policies has emerged as a popular tool both in theory and practice.
By Alex Davey, Alena Shilova, Brahim Driss, Riad Akrour
The paper introduces BUMEX, a reinforcement learning exploration strategy that leverages a set of prior models containing the true transition kernel and reward function. By optimizing over this model set, the method derives upper and lower bounds on the Q‑function to guide exploration, providing theoretical guarantees of convergence to the optimal policy. When the model set follows a bounded‑parameter MDP structure, the optimization becomes convex, enabling finite‑time convergence under mild assumptions and demonstrating accelerated learning in simulations.
By J. S. van Hulst, W. P. M. H. Heemels, D. J. Antunes
arXiv:2510. 03494v2 Announce Type: replace Abstract: We study finite-horizon offline reinforcement learning (RL) with function approximation for both policy evaluation and policy optimization.
By Volodymyr Tkachuk, Csaba Szepesv\'ari, Xiaoqi Tan