arXiv Machine Learning August 24, 2026 By J. S. van Hulst, W. P. M. H. Heemels, D. J. Antunes

Smart Exploration in Reinforcement Learning using Bounded Uncertainty Models

Read the original on arXiv Machine Learning →

The paper introduces BUMEX, a reinforcement learning exploration strategy that leverages a set of prior models containing the true transition kernel and reward function. By optimizing over this model set, the method derives upper and lower bounds on the Q‑function to guide exploration, providing theoretical guarantees of convergence to the optimal policy. When the model set follows a bounded‑parameter MDP structure, the optimization becomes convex, enabling finite‑time convergence under mild assumptions and demonstrating accelerated learning in simulations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 14

Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning

arXiv:2506. 07040v4 Announce Type: replace-cross Abstract: We study model-free methods for distributionally robust infinite-horizon average-reward Markov decision processes (MDPs).

By Yang Xu, Swetha Ganesh, Vaneet Aggarwal

reinforcement-learning

arXiv AI

Jun 18

Robust Regularized Policy Iteration under Transition Uncertainty

arXiv:2603. 09344v3 Announce Type: replace Abstract: Offline reinforcement learning (RL) enables data-efficient and safe policy learning without online exploration, but its performance often degrades under distribution shift.

By Hongqiang Lin, Zhenghui Fu, Weihao Tang, Pengfei Wang, Yiding Sun, Qixian Huang, Dongxu Zhang

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 30

Theoretical Foundations and Effective Algorithms for Policy-Aware Simulator Learning

arXiv:2605. 29032v2 Announce Type: replace Abstract: Model-based reinforcement learning (MBRL) agents typically learn world models by minimizing predictive loss.

By Christoph Dann, Yishay Mansour, Mehryar Mohri

agents reinforcement-learning safety

arXiv Machine Learning

Jun 25

Auto-exploration for online reinforcement learning

arXiv:2512. 06244v2 Announce Type: replace Abstract: The exploration-exploitation dilemma in reinforcement learning (RL) is a fundamental challenge to efficient RL algorithms.

By Caleb Ju, Guanghui Lan

reinforcement-learning

arXiv AI

Jun 16

Deep Q-Learning on H\"older Spaces

arXiv:2606. 16846v1 Announce Type: cross Abstract: We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions.

By Qian Qi

diffusion reinforcement-learning

arXiv AI

Jun 30

Exploration and Online Transfer with Behavioral Foundation Models

arXiv:2606. 29980v1 Announce Type: new Abstract: Zero-shot Transfer in Reinforcement Learning (RL) aims to train an agent that can generate optimal policies for any reward function, without additional learning at transfer time, while training only on reward-free trajectories.

By Louis Bagot (SyCoSMA), Mathieu Lefort (LIRIS, SyCoSMA, IRISA, MALT, UR), La\"etitia Matignon (SyCoSMA)

agents reinforcement-learning

One email a morning, machine-written

Related stories

Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning

Robust Regularized Policy Iteration under Transition Uncertainty

Theoretical Foundations and Effective Algorithms for Policy-Aware Simulator Learning

Auto-exploration for online reinforcement learning

Deep Q-Learning on H\"older Spaces

Exploration and Online Transfer with Behavioral Foundation Models