arXiv AI
Aug 11
Switching Theory for Q-Learning
arXiv:2604. 19569v5 Announce Type: replace-cross Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.
Spectral Analysis of Dueling Q-Learning
arXiv:2607. 08340v1 Announce Type: cross Abstract: Q-learning is a fundamental algorithm in reinforcement learning (RL) for solving discounted Markov decision processes (MDPs) when the transition kernel is unknown.
Related stories
arXiv Machine Learning
Jul 7
A Switching System Theory of Q-Learning with Linear Function Approximation
arXiv:2605. 11021v3 Announce Type: replace Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.
arXiv AI
Jun 3
Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics
arXiv:2606. 02645v1 Announce Type: cross Abstract: Periodic target updates in Q-learning and soft target updates in actor-critic methods are empirically well established stabilization mechanisms, but their precise theoretical explanation is still incomplete.
arXiv AI
Jun 26
Heavy-Ball Q-Learning with Residual Weighting Correction
arXiv:2606. 27112v1 Announce Type: cross Abstract: This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence.
arXiv AI
Jul 1
Lyapunov-Certified Direct Switching Theory for Q-Learning
arXiv:2604. 19569v4 Announce Type: replace-cross Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.
arXiv AI
Jul 7
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
arXiv:2605. 16103v2 Announce Type: replace Abstract: Q-learning is known to suffer from overestimation bias: because the Bellman update maximizes noisy or imperfect action-value estimates, positive errors can be selected and propagated, causing learned values to exceed the true optimal values.
arXiv AI
Jul 17
Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis
arXiv:2503. 18607v2 Announce Type: replace-cross Abstract: We introduce the Switching Non-Stationary Markov Decision Process (SNS-MDP) framework, in which the environment transitions among a finite set of MDPs governed by a latent Markov chain while the agent observes only the external state.
Hugging Face Trending Papers
Jun 25
Heavy-Ball Q-Learning with Residual Weighting Correction
This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence. It also identifies conditions under which the method is theoretically guaranteed to converge faster than standard Q-learning.
arXiv AI
Jun 16
Deep Q-Learning on H\"older Spaces
arXiv:2606. 16846v1 Announce Type: cross Abstract: We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions.
arXiv Machine Learning
Aug 4
Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning
arXiv:2608. 02034v1 Announce Type: new Abstract: Multi-step returns accelerate reward propagation in off-policy reinforcement learning, but couple the evaluation of each decision to the suboptimal logged actions that follow it, inducing a pessimistic bias that grows with the horizon.
arXiv AI
Aug 3
StaQ: a Finite Memory Approach to Discrete Action Policy Mirror Descent
arXiv:2506. 13862v2 Announce Type: replace-cross Abstract: In Reinforcement Learning (RL), regularization with a Kullback-Leibler divergence that penalizes large deviations between successive policies has emerged as a popular tool both in theory and practice.
arXiv Machine Learning
5d ago
Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection
arXiv:2608. 12912v1 Announce Type: new Abstract: This paper considers the overestimation bias problem of Q-learning in the setting of a large action space, for the purpose of relieving the bottleneck of existing methods.