arXiv Machine Learning August 4, 2026 By Abdelghani Ghanem, Mounir Ghogho

Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning

Read the original on arXiv Machine Learning →

arXiv:2608. 02034v1 Announce Type: new Abstract: Multi-step returns accelerate reward propagation in off-policy reinforcement learning, but couple the evaluation of each decision to the suboptimal logged actions that follow it, inducing a pessimistic bias that grows with the horizon.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 7

Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning

arXiv:2605. 16103v2 Announce Type: replace Abstract: Q-learning is known to suffer from overestimation bias: because the Bellman update maximizes noisy or imperfect action-value estimates, positive errors can be selected and propagated, causing learned values to exceed the true optimal values.

By Donghwan Lee

reinforcement-learning safety

arXiv Machine Learning

Aug 5

Revisiting TD Target Aggregation under Uncertainty in Q-Learning

arXiv:2608. 03069v1 Announce Type: new Abstract: Deep Q-Networks (DQNs) learn value functions through bootstrapped temporal-difference updates, where future returns are approximated using a greedy maximization over next-state action values.

By Lipeng Zu, Xiaonan Zhang

reinforcement-learning benchmarks

arXiv AI

Jul 10

Spectral Analysis of Dueling Q-Learning

arXiv:2607. 08340v1 Announce Type: cross Abstract: Q-learning is a fundamental algorithm in reinforcement learning (RL) for solving discounted Markov decision processes (MDPs) when the transition kernel is unknown.

By Donghwan Lee

reinforcement-learning

arXiv AI

1d ago

Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning

arXiv:2608. 15088v1 Announce Type: cross Abstract: Human-in-the-loop (HIL) online reinforcement learning for real robots must absorb human interventions quickly while continuing to improve beyond the human prior.

By Zihang Wang, Yishan Wang

agents reinforcement-learning robotics

arXiv Machine Learning

5d ago

Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection

arXiv:2608. 12912v1 Announce Type: new Abstract: This paper considers the overestimation bias problem of Q-learning in the setting of a large action space, for the purpose of relieving the bottleneck of existing methods.

By Pu Li, Tao Tan, Hong Xie, Xiaoyu Shi, Mingsheng Shang

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jul 10

Beyond Success Rates: Trainability and Extractability for Offline GCRL

arXiv:2602. 05459v2 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) is typically benchmarked by the best tuned success rate of each method.

By Jan Malte T\"opperwien, Aditya Mohan, Marius Lindauer

reinforcement-learning robotics

Related stories

Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning

Revisiting TD Target Aggregation under Uncertainty in Q-Learning

Spectral Analysis of Dueling Q-Learning

Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning

Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection

Beyond Success Rates: Trainability and Extractability for Offline GCRL