arXiv Machine Learning
Non-asymptotic Convergence of Average-reward Q-learning with Options
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 7
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
arXiv:2605. 16103v2 Announce Type: replace Abstract: Q-learning is known to suffer from overestimation bias: because the Bellman update maximizes noisy or imperfect action-value estimates, positive errors can be selected and propagated, causing learned values to exceed the true optimal values.
arXiv AI
Jun 16
Deep Q-Learning on H\"older Spaces
arXiv:2606. 16846v1 Announce Type: cross Abstract: We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions.
Hugging Face Trending Papers
Sep 23
Limiting-Kernel Q($λ$): Bridging Short and Long Horizons
In value-based reinforcement learning, improving the accuracy of policy evaluation has been shown to improve downstream policy optimization performance. The widely adopted family of approximations rel...
Hugging Face Trending Papers
Oct 15
V-SHARE: Provable Convergence Guarantees for Advantage-based Algorithms
In reinforcement learning, advantage-based value-learning methods, unlike standard Q-learning, share information across actions by decomposing the action-value function into a state-dependent value co...
arXiv Machine Learning
Sep 24
Limiting-Kernel Q($\lambda$): Bridging Short and Long Horizons
Limiting‑Kernel Q(λ) (LKQL) is an off‑policy value estimator that blends n‑step truncation with a long‑horizon approximation based on the limiting kernel. It maintains the computational efficiency of n‑step methods while improving policy evaluation accuracy, especially for long‑horizon tasks. The authors prove faster convergence of LKQL’s operator under aperiodicity and near‑on‑policy conditions, and demonstrate empirical gains on MuJoCo continuous‑control benchmarks.
arXiv AI
Oct 16
V-SHARE: Provable Convergence Guarantees for Advantage-based Algorithms
V‑SHARE is an advantage‑based value‑learning algorithm that blends cross‑action value sharing with an importance‑weighted Bellman‑residual correction. The paper presents the first full theoretical analysis of such an algorithm, offering both asymptotic convergence and finite‑sample guarantees. It also introduces schedules that balance deterministic contraction against importance‑weighting noise, and a two‑timescale analysis that eliminates a residual‑weight threshold while maintaining convergence.