An Introduction to Q-Learning Part 2/2

An Introduction to Q-Learning Part 1

Equivalence between policy gradients and soft Q-learning

Heavy-Ball Q-Learning with Residual Weighting Correction

arXiv:2606. 27112v1 Announce Type: cross Abstract: This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence.

Spectral Analysis of Dueling Q-Learning

arXiv:2607. 08340v1 Announce Type: cross Abstract: Q-learning is a fundamental algorithm in reinforcement learning (RL) for solving discounted Markov decision processes (MDPs) when the transition kernel is unknown.

Pareto Q-Learning with Reward Machines

arXiv:2606. 19134v1 Announce Type: cross Abstract: We present Pareto Q-Learning with Reward Machines (PQLRM), a multi-objective reinforcement learning algorithm for tasks whose reward structure is specified by a set of reward machines (RMs).

From Classical to Quantum Reinforcement Learning and Its Applications in Quantum Control: A Beginner's Tutorial

arXiv:2601. 08662v3 Announce Type: replace Abstract: This tutorial is designed to make reinforcement learning (RL) more accessible to undergraduate students by offering clear, example-driven explanations.

UCB exploration via Q-ensembles

A Switching System Theory of Q-Learning with Linear Function Approximation

arXiv:2605. 11021v3 Announce Type: replace Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.

Quantum vs. Classical Machine Learning: A Unified Empirical Comparison

arXiv:2607. 01197v1 Announce Type: new Abstract: Quantum computing has emerged as a promising computational paradigm for machine learning (ML), with the potential to offer computational advantages over classical approaches.

Deep Q-Learning on H\"older Spaces

arXiv:2606. 16846v1 Announce Type: cross Abstract: We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions.

Smart Exploration in Reinforcement Learning using Bounded Uncertainty Models

The paper introduces BUMEX, a reinforcement learning exploration strategy that leverages a set of prior models containing the true transition kernel and reward function. By optimizing over this model set, the method derives upper and lower bounds on the Q‑function to guide exploration, providing theoretical guarantees of convergence to the optimal policy. When the model set follows a bounded‑parameter MDP structure, the optimization becomes convex, enabling finite‑time convergence under mild assumptions and demonstrating accelerated learning in simulations.

An Introduction to Deep Reinforcement Learning