arXiv Machine Learning June 9, 2026 By Wei-Cheng Lee, Francesco Orabona

A Robust $\widetilde{\mathcal{O}}(1/\sqrt{T})$ Rate for Unprojected TD Learning with Linear Function Approximation

Read the original on arXiv Machine Learning →

arXiv:2506. 01052v3 Announce Type: replace Abstract: We investigate the finite-time convergence properties of Temporal Difference (TD) learning with linear function approximation, a cornerstone of reinforcement learning.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

reinforcement-learning

arXiv Machine Learning

Jun 5

Fast and Robust Convergence Rate for TD(0) with Linear Function Approximation, Universal Learning Steps and I.I.D. Samples

arXiv:2606. 05967v1 Announce Type: cross Abstract: In this paper, we study the finite-time behavior of the TD(0) temporal-difference method with linear function approximation (LFA).

By Ziad Kobeissi (L2S), \'Elo\"ise Berthier (U2IS)

Hugging Face Trending Papers

Jun 4

Fast and Robust Convergence Rate for TD(0) with Linear Function Approximation, Universal Learning Steps and I.I.D. Samples

In this paper, we study the finite-time behavior of the TD(0) temporal-difference method with linear function approximation (LFA). We consider on-policy independent and identically distributed (i.

arXiv AI

Jul 14

Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning

arXiv:2506. 07040v4 Announce Type: replace-cross Abstract: We study model-free methods for distributionally robust infinite-horizon average-reward Markov decision processes (MDPs).

By Yang Xu, Swetha Ganesh, Vaneet Aggarwal

reinforcement-learning

arXiv Machine Learning

Jun 11

Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity

arXiv:2606. 11431v1 Announce Type: new Abstract: Mirror Descent (MD) extends Gradient Descent (GD) beyond Euclidean geometry and has recently reappeared as a lens for KL-regularized policy optimization in reinforcement learning and LLM post-training.

By Shira Vansover-Hager, Matan Schliserman, Ofir Schlisselberg, Tomer Koren

llms reinforcement-learning

arXiv Machine Learning

Jul 28

Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes

arXiv:2607. 22982v1 Announce Type: new Abstract: Natural Policy Gradient (NPG) is a well-established Reinforcement Learning algorithm that underlies widely used methods such as Trust Region Policy Optimization and Proximal Policy Optimization, both of which have demonstrated strong empirical success.

By Asha Barua, Sajad Khodadadian

reinforcement-learning

Hugging Face Trending Papers

Aug 3

Finite-Time Analysis of Discounted Exponential-Utility Reinforcement Learning

Discounted exponential utility provides a principled criterion for risk-sensitive sequential decision-making, but its nonlinear structure complicates reinforcement learning. A recent work \citep{thoppe2026reinforcement} addressed this difficulty by introducing a Bellman-compatible surrogate and two model-free fixed-point algorithms for optimizing it over stationary policies.

reinforcement-learning

Related stories

Fast and Robust Convergence Rate for TD(0) with Linear Function Approximation, Universal Learning Steps and I.I.D. Samples

Fast and Robust Convergence Rate for TD(0) with Linear Function Approximation, Universal Learning Steps and I.I.D. Samples

Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning

Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity

Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes

Finite-Time Analysis of Discounted Exponential-Utility Reinforcement Learning