arXiv Machine Learning
Jul 7
A Switching System Theory of Q-Learning with Linear Function Approximation
arXiv:2605. 11021v3 Announce Type: replace Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.
Switching Theory for Q-Learning
arXiv:2604. 19569v5 Announce Type: replace-cross Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.
Related stories
arXiv AI
Jul 1
Lyapunov-Certified Direct Switching Theory for Q-Learning
arXiv:2604. 19569v4 Announce Type: replace-cross Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.
arXiv AI
Jul 10
Spectral Analysis of Dueling Q-Learning
arXiv:2607. 08340v1 Announce Type: cross Abstract: Q-learning is a fundamental algorithm in reinforcement learning (RL) for solving discounted Markov decision processes (MDPs) when the transition kernel is unknown.
arXiv AI
Jun 3
Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics
arXiv:2606. 02645v1 Announce Type: cross Abstract: Periodic target updates in Q-learning and soft target updates in actor-critic methods are empirically well established stabilization mechanisms, but their precise theoretical explanation is still incomplete.
arXiv AI
Jun 26
Heavy-Ball Q-Learning with Residual Weighting Correction
arXiv:2606. 27112v1 Announce Type: cross Abstract: This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence.
arXiv AI
Jul 7
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
arXiv:2605. 16103v2 Announce Type: replace Abstract: Q-learning is known to suffer from overestimation bias: because the Bellman update maximizes noisy or imperfect action-value estimates, positive errors can be selected and propagated, causing learned values to exceed the true optimal values.
Hugging Face Trending Papers
Jun 25
Heavy-Ball Q-Learning with Residual Weighting Correction
This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence. It also identifies conditions under which the method is theoretically guaranteed to converge faster than standard Q-learning.
arXiv AI
Jul 17
Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis
arXiv:2503. 18607v2 Announce Type: replace-cross Abstract: We introduce the Switching Non-Stationary Markov Decision Process (SNS-MDP) framework, in which the environment transitions among a finite set of MDPs governed by a latent Markov chain while the agent observes only the external state.
arXiv Machine Learning
Aug 4
Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning
arXiv:2608. 02034v1 Announce Type: new Abstract: Multi-step returns accelerate reward propagation in off-policy reinforcement learning, but couple the evaluation of each decision to the suboptimal logged actions that follow it, inducing a pessimistic bias that grows with the horizon.
arXiv AI
Jun 10
Geometrically Averaged Hard Target Updates for Linear Q-Learning
arXiv:2606. 10835v1 Announce Type: cross Abstract: Periodic hard target updates are among the most common stabilization devices in modern deep Q-learning.
arXiv Machine Learning
Sep 22
Optimal Sample Complexity of Stable Discounted Markov Decision Processes
arXiv:2302.07477v4 Announce Type: replace Abstract: We study the optimal sample complexity of tabular reinforcement learning for infinite-horizon discounted Markov decision processes. The unrestricte...
arXiv Machine Learning
Sep 24
Limiting-Kernel Q($\lambda$): Bridging Short and Long Horizons
Limiting‑Kernel Q(λ) (LKQL) is an off‑policy value estimator that blends n‑step truncation with a long‑horizon approximation based on the limiting kernel. It maintains the computational efficiency of n‑step methods while improving policy evaluation accuracy, especially for long‑horizon tasks. The authors prove faster convergence of LKQL’s operator under aperiodicity and near‑on‑policy conditions, and demonstrate empirical gains on MuJoCo continuous‑control benchmarks.