arXiv Machine Learning
Aug 11
Regret of exploratory policy improvement and $q$-learning
arXiv:2411. 01302v2 Announce Type: replace Abstract: We study the convergence of $q$-learning and related algorithms introduced by Jia and Zhou (J.
Equivalence between policy gradients and soft Q-learning
Related stories
arXiv AI
Jul 20
Soft $Q(\lambda)$: A multi-step off-policy method for entropy regularised reinforcement learning using eligibility traces
arXiv:2604. 13780v2 Announce Type: replace-cross Abstract: Soft Q-learning has emerged as a versatile model-free method for entropy-regularised reinforcement learning, optimising for returns augmented with a penalty on the divergence from a reference policy.
arXiv AI
Jun 16
Deep Q-Learning on H\"older Spaces
arXiv:2606. 16846v1 Announce Type: cross Abstract: We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions.
arXiv AI
Jun 19
Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic Methods
arXiv:2601. 22970v2 Announce Type: replace-cross Abstract: Policies learned via continuous actor-critic methods often exhibit erratic, high-frequency oscillations, making them unsuitable for physical deployment.
arXiv AI
Jun 18
Pareto Q-Learning with Reward Machines
arXiv:2606. 19134v1 Announce Type: cross Abstract: We present Pareto Q-Learning with Reward Machines (PQLRM), a multi-objective reinforcement learning algorithm for tasks whose reward structure is specified by a set of reward machines (RMs).
arXiv AI
Jul 10
Spectral Analysis of Dueling Q-Learning
arXiv:2607. 08340v1 Announce Type: cross Abstract: Q-learning is a fundamental algorithm in reinforcement learning (RL) for solving discounted Markov decision processes (MDPs) when the transition kernel is unknown.
arXiv AI
Jun 3
Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics
arXiv:2606. 02645v1 Announce Type: cross Abstract: Periodic target updates in Q-learning and soft target updates in actor-critic methods are empirically well established stabilization mechanisms, but their precise theoretical explanation is still incomplete.
arXiv AI
Jun 26
Heavy-Ball Q-Learning with Residual Weighting Correction
arXiv:2606. 27112v1 Announce Type: cross Abstract: This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence.
arXiv AI
Jul 7
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
arXiv:2605. 16103v2 Announce Type: replace Abstract: Q-learning is known to suffer from overestimation bias: because the Bellman update maximizes noisy or imperfect action-value estimates, positive errors can be selected and propagated, causing learned values to exceed the true optimal values.
Hugging Face Trending Papers
Jun 25
Heavy-Ball Q-Learning with Residual Weighting Correction
This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence. It also identifies conditions under which the method is theoretically guaranteed to converge faster than standard Q-learning.
arXiv Machine Learning
Jul 22
Gaussian-Mixture-Model Q-Functions for Policy Iteration in Reinforcement Learning
arXiv:2512. 18763v2 Announce Type: replace Abstract: Unlike their conventional use as estimators of probability density functions in reinforcement learning (RL), this paper introduces a novel function-approximation role for Gaussian mixture models (GMMs) as direct surrogates for Q-function losses.
arXiv AI
Aug 3
StaQ: a Finite Memory Approach to Discrete Action Policy Mirror Descent
arXiv:2506. 13862v2 Announce Type: replace-cross Abstract: In Reinforcement Learning (RL), regularization with a Kullback-Leibler divergence that penalizes large deviations between successive policies has emerged as a popular tool both in theory and practice.