arXiv Machine Learning August 21, 2026 By Shangtong Zhang

Towards Formalizing Reinforcement Learning Theory: A Robbins-Siegmund Approach

Read the original on arXiv Machine Learning →

arXiv:2511. 03618v2 Announce Type: replace Abstract: In this paper, we formalize the almost sure convergence of $Q$-learning and linear temporal difference (TD) learning with Markovian samples using the Lean 4 theorem prover based on the Mathlib library.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

reinforcement-learning

arXiv Machine Learning

Jul 1

End-to-End Efficient RL for Linear Bellman Complete MDPs with Deterministic Transitions

arXiv:2603. 23461v2 Announce Type: replace Abstract: We study reinforcement learning (RL) with linear function approximation in Markov Decision Processes (MDPs) satisfying \emph{linear Bellman completeness} -- a fundamental setting where the Bellman backup of any linear value function remains linear.

By Zakaria Mhammedi, Alexander Rakhlin, Nneka Okolo

reinforcement-learning

arXiv Machine Learning

Aug 11

Regret of exploratory policy improvement and $q$-learning

arXiv:2411. 01302v2 Announce Type: replace Abstract: We study the convergence of $q$-learning and related algorithms introduced by Jia and Zhou (J.

By Wenpin Tang, Xun Yu Zhou

diffusion reinforcement-learning

arXiv Machine Learning

Oct 5

On the Convergence of Success Conditioning for Policy Optimization

arXiv:2610.03642v1 Announce Type: new Abstract: Success conditioning is a strategy for improving decision-making policies in stochastic environments; it updates a policy by increasing the probability...

By Matthew Brun, Xu Andy Sun

reinforcement-learning

arXiv Machine Learning

Oct 15

Non-asymptotic Convergence of Average-reward Q-learning with Options

arXiv:2610.20347v1 Announce Type: new Abstract: We study the finite-sample convergence of Q-learning algorithms for the average-reward formulation of Reinforcement Learning (RL) with options. While o...

By Kintan Saha, Ahana Deb, Anders Jonsson, Debabrota Basu

reinforcement-learning

arXiv AI

Jul 17

Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis

arXiv:2503. 18607v2 Announce Type: replace-cross Abstract: We introduce the Switching Non-Stationary Markov Decision Process (SNS-MDP) framework, in which the environment transitions among a finite set of MDPs governed by a latent Markov chain while the agent observes only the external state.

By Mohsen Amiri, Sindri Magn\'usson

agents reinforcement-learning

arXiv Machine Learning

Jun 2

Trajectory Data Suffices for Statistically Efficient Policy Evaluation in Fixed-Horizon Offline RL with Linear $q^\pi$-Realizability and Concentrability

arXiv:2510. 03494v2 Announce Type: replace Abstract: We study finite-horizon offline reinforcement learning (RL) with function approximation for both policy evaluation and policy optimization.

By Volodymyr Tkachuk, Csaba Szepesv\'ari, Xiaoqi Tan

reinforcement-learning

One email a morning, machine-written

Related stories

End-to-End Efficient RL for Linear Bellman Complete MDPs with Deterministic Transitions

Regret of exploratory policy improvement and $q$-learning

On the Convergence of Success Conditioning for Policy Optimization

Non-asymptotic Convergence of Average-reward Q-learning with Options

Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis

Trajectory Data Suffices for Statistically Efficient Policy Evaluation in Fixed-Horizon Offline RL with Linear $q^\pi$-Realizability and Concentrability