arXiv AI
Jun 30
Accelerating Q-learning through Efficient Value-Sharing across Actions
arXiv:2606. 29806v1 Announce Type: cross Abstract: Action-values are foundational to many control algorithms such as Q-learning.
On BatchNorm Forward Modes in Value-Based Reinforcement Learning
Related stories
arXiv Machine Learning
Sep 22
Streaming Deep Reinforcement Learning Finally Works
arXiv:2410.14606v3 Announce Type: replace Abstract: Learning from a stream of experience as it arrives, also known as streaming learning, is a core part of natural learning. However, reliable streami...
arXiv AI
Aug 10
Aftab: A Comprehensive Benchmark of CNN Encoders and Advanced Value Functions in Parallelized Q-Networks
arXiv:2608. 07335v1 Announce Type: cross Abstract: Recent advancements in deep reinforcement learning have increasingly favored simplified, highly parallelized paradigms.
arXiv Machine Learning
5d ago
From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning
The paper introduces Q-Target Pretrained Transformers (QTPT), a method that replaces supervised behavior cloning with a Bellman-style Q‑target objective for in‑context reinforcement learning. QTPT retains the context‑conditioned Transformer architecture but learns to estimate action values using rewards and transitions from the context, rather than merely imitating offline actions. The authors provide theoretical analysis in stochastic linear bandits and finite‑horizon MDPs, demonstrating improved robustness to weak or suboptimal data, and empirically show gains over supervised pretraining on controlled RL benchmarks and extensions to D4RL Kitchen and AntMaze.
arXiv AI
Aug 18
Understanding and Stabilizing Deep Q-Learning via Controlled Bootstrapping and Regulated Value Dynamics
arXiv:2608. 16182v1 Announce Type: cross Abstract: Deep Q-learning (DQL) has achieved remarkable empirical success in reinforcement learning, yet its training process remains notoriously unstable.
arXiv AI
Aug 3
StaQ: a Finite Memory Approach to Discrete Action Policy Mirror Descent
arXiv:2506. 13862v2 Announce Type: replace-cross Abstract: In Reinforcement Learning (RL), regularization with a Kullback-Leibler divergence that penalizes large deviations between successive policies has emerged as a popular tool both in theory and practice.
arXiv Machine Learning
Jul 24
Offline RL with Hierarchical Action Chunking
arXiv:2607. 20834v1 Announce Type: new Abstract: Offline goal-conditioned reinforcement learning (RL) holds the promise of learning general-purpose policies from static datasets.
arXiv Machine Learning
Aug 5
Revisiting TD Target Aggregation under Uncertainty in Q-Learning
arXiv:2608. 03069v1 Announce Type: new Abstract: Deep Q-Networks (DQNs) learn value functions through bootstrapped temporal-difference updates, where future returns are approximated using a greedy maximization over next-state action values.
arXiv AI
Jun 4
Unifying Model-Free Efficiency and Model-Based Representations via Latent Dynamics
arXiv:2602. 12643v2 Announce Type: replace-cross Abstract: We present Unified Latent Dynamics (ULD), a novel reinforcement learning algorithm that unifies the efficiency of model-free methods with the representational strengths of model-based approaches, without incurring planning overhead.
arXiv Machine Learning
Jun 5
Scalable Reinforcement Learning via Adaptive Batch Scaling
arXiv:2605. 21557v2 Announce Type: replace-cross Abstract: Conventional wisdom holds that large-batch training is fundamentally incompatible with Reinforcement Learning (RL) - beyond a modest threshold, increasing batch sizes typically yields diminishing returns or performance degradation due to the inherent non-stationarity of the data distribution.
arXiv Machine Learning
Jul 23
Memory Merge DQN: Sensitivity Weighted Target Updates for Stable Value Learning
arXiv:2607. 19397v1 Announce Type: new Abstract: Deep Q-networks use target networks to stabilise bootstrapped value learning, but the standard hard copy update also introduces a tradeoff.
arXiv Machine Learning
Jun 19
Approximate Next Policy Sampling: Replacing Conservative Target Policy Updates in Deep RL
arXiv:2605. 05481v2 Announce Type: replace Abstract: We revisit a classic "chicken-and-egg" problem in reinforcement learning: to safely improve a policy, the value function must be accurate on the state-visitation distribution of the updated policy.