arXiv Machine Learning August 21, 2026 By Shangtong Zhang

Towards Formalizing Reinforcement Learning Theory: A Robbins-Siegmund Approach

Read the original on arXiv Machine Learning →

arXiv:2511. 03618v2 Announce Type: replace Abstract: In this paper, we formalize the almost sure convergence of $Q$-learning and linear temporal difference (TD) learning with Markovian samples using the Lean 4 theorem prover based on the Mathlib library.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

reinforcement-learning

arXiv Machine Learning

Jul 1

End-to-End Efficient RL for Linear Bellman Complete MDPs with Deterministic Transitions

arXiv:2603. 23461v2 Announce Type: replace Abstract: We study reinforcement learning (RL) with linear function approximation in Markov Decision Processes (MDPs) satisfying \emph{linear Bellman completeness} -- a fundamental setting where the Bellman backup of any linear value function remains linear.

By Zakaria Mhammedi, Alexander Rakhlin, Nneka Okolo

reinforcement-learning

arXiv Machine Learning

Aug 11

Regret of exploratory policy improvement and $q$-learning

arXiv:2411. 01302v2 Announce Type: replace Abstract: We study the convergence of $q$-learning and related algorithms introduced by Jia and Zhou (J.

By Wenpin Tang, Xun Yu Zhou

diffusion reinforcement-learning

arXiv AI

Jul 17

Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis

arXiv:2503. 18607v2 Announce Type: replace-cross Abstract: We introduce the Switching Non-Stationary Markov Decision Process (SNS-MDP) framework, in which the environment transitions among a finite set of MDPs governed by a latent Markov chain while the agent observes only the external state.

By Mohsen Amiri, Sindri Magn\'usson

agents reinforcement-learning

arXiv Machine Learning

Jun 2

Trajectory Data Suffices for Statistically Efficient Policy Evaluation in Fixed-Horizon Offline RL with Linear $q^\pi$-Realizability and Concentrability

arXiv:2510. 03494v2 Announce Type: replace Abstract: We study finite-horizon offline reinforcement learning (RL) with function approximation for both policy evaluation and policy optimization.

By Volodymyr Tkachuk, Csaba Szepesv\'ari, Xiaoqi Tan

reinforcement-learning

arXiv Machine Learning

Aug 3

Commit to the Bit: Reactive Reinforcement Learning Done Right

arXiv:2605. 28276v2 Announce Type: replace Abstract: Reinforcement learning algorithms are commonly analyzed (and designed) under the Markov assumption.

By Onno Eberhard, Claire Vernade, Michael Muehlebach

agents reinforcement-learning

arXiv Machine Learning

Jul 21

Non-Asymptotic Best Policy Identification Guarantees in Online Reinforcement Learning

arXiv:2607. 17201v1 Announce Type: cross Abstract: In this work we study the Best Policy Identification (BPI) problem in online, tabular Reinforcement Learning.

By Joseph Lazzaro, Alessio Russo, Aldo Pacchiano

agents reinforcement-learning

One email a morning, machine-written

Related stories

End-to-End Efficient RL for Linear Bellman Complete MDPs with Deterministic Transitions

Regret of exploratory policy improvement and $q$-learning

Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis

Trajectory Data Suffices for Statistically Efficient Policy Evaluation in Fixed-Horizon Offline RL with Linear $q^\pi$-Realizability and Concentrability

Commit to the Bit: Reactive Reinforcement Learning Done Right

Non-Asymptotic Best Policy Identification Guarantees in Online Reinforcement Learning