arXiv AI

A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learning

The paper presents a three‑phase study of visual encoders and value‑estimation methods for replay‑free parallelized Q‑learning within the Parallelized Q‑Network framework. It compares eight convolutional encoder topologies, adds Hadamax‑style multiplicative interactions and pooling, and evaluates categorical‑dueling, ensemble‑dueling, and categorical ensemble‑dueling value‑estimation configurations. The resulting architecture, Aftab, outperforms a baseline PQN on Atari‑57 and shows improved performance on Procgen Hard, demonstrating that visual topology, multiplicative representation, and downstream value‑estimation design significantly influence replay‑free Q‑learning when considered alongside computational complexity.

arXiv AI

Sep 25

Aftab: A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learning

The paper presents Aftab, a new architecture for replay‑free parallelized Q‑learning that systematically explores visual encoder designs, multiplicative feature interactions, and value‑estimation strategies. Through a three‑phase study on Atari‑57, the authors compare eight convolutional encoders, integrate Hadamax‑style interactions, and evaluate categorical‑dueling, ensemble‑dueling, and combined configurations, ultimately achieving a higher human‑normalized score than the baseline PQN. Aftab is also evaluated on Procgen Hard, showing improved terminal IQM and learning‑curve area, and the full framework is released as open source.

By Taha Shieenavaz, Shabnam Zareshahraki, Loris Nanni

reinforcement-learning

arXiv AI

Aug 10

Aftab: A Comprehensive Benchmark of CNN Encoders and Advanced Value Functions in Parallelized Q-Networks

arXiv:2608. 07335v1 Announce Type: cross Abstract: Recent advancements in deep reinforcement learning have increasingly favored simplified, highly parallelized paradigms.

By Taha Shieenavaz, Shabnam Zareshahraki, Loris Nanni

reinforcement-learning benchmarks

arXiv Machine Learning

Jul 23

Memory Merge DQN: Sensitivity Weighted Target Updates for Stable Value Learning

arXiv:2607. 19397v1 Announce Type: new Abstract: Deep Q-networks use target networks to stabilise bootstrapped value learning, but the standard hard copy update also introduces a tradeoff.

By Adrian Ly, Richard Dazeley, Peter Vamplew, Sunil Aryal, Francisco Cruz

agents

arXiv Machine Learning

Sep 22

Streaming Deep Reinforcement Learning Finally Works

arXiv:2410.14606v3 Announce Type: replace Abstract: Learning from a stream of experience as it arrives, also known as streaming learning, is a core part of natural learning. However, reliable streami...

By Mohamed Elsayed, Elena Sorina Lupu, Gautham Vasan, A. Rupam Mahmood

reinforcement-learning robotics benchmarks

arXiv Machine Learning

Aug 4

Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning

arXiv:2608. 02034v1 Announce Type: new Abstract: Multi-step returns accelerate reward propagation in off-policy reinforcement learning, but couple the evaluation of each decision to the suboptimal logged actions that follow it, inducing a pessimistic bias that grows with the horizon.

By Abdelghani Ghanem, Mounir Ghogho

reinforcement-learning robotics efficiency safety

arXiv AI

Sep 10

On BatchNorm Forward Modes in Value-Based Reinforcement Learning

arXiv:2609.06421v1 Announce Type: cross Abstract: Batch normalization (BN) substantially improves sample efficiency in continuous-control actor-critic methods such as CrossQ, yet recent studies repor...

By Daniel Palenicek, Mikael Henaff, Scott Fujimoto, Koustuv Sinha

reinforcement-learning

arXiv AI

Jun 4

Unifying Model-Free Efficiency and Model-Based Representations via Latent Dynamics

arXiv:2602. 12643v2 Announce Type: replace-cross Abstract: We present Unified Latent Dynamics (ULD), a novel reinforcement learning algorithm that unifies the efficiency of model-free methods with the representational strengths of model-based approaches, without incurring planning overhead.

By Jashaswimalya Acharjee, Balaraman Ravindran

rag diffusion reinforcement-learning robotics

arXiv AI

Jun 30

TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning

arXiv:2401. 11512v2 Announce Type: replace-cross Abstract: Identifying the most suitable variables to represent the state is a fundamental challenge in Reinforcement Learning (RL).

By Charles Westphal, Stephen Hailes, Mirco Musolesi

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jul 7

Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Control

arXiv:2607. 04978v1 Announce Type: new Abstract: Joint-Embedding Predictive Architectures (JEPAs) underpin a growing family of latent world models for control from raw pixels, but every existing JEPA world model commits at training time to a single inference paradigm: either trajectory optimisation in a learned dynamics model, or direct behaviour cloning.

By Ruslan Rakhimov, George Bredis, Yuriy Maksyuta, Daniil Gavrilov

rag diffusion benchmarks

arXiv AI

Jun 12

SymQNet: Amortized Acquisition for Low-Latency Adaptive Hamiltonian Learning

arXiv:2606. 12808v1 Announce Type: cross Abstract: Adaptive Hamiltonian learning is central to calibrating and characterizing quantum devices.

By Yash Vardhan Tomar, Dheeraj Peddireddy, Vaneet Aggarwal

benchmarks

arXiv Machine Learning

Aug 14

Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection

arXiv:2608. 12912v1 Announce Type: new Abstract: This paper considers the overestimation bias problem of Q-learning in the setting of a large action space, for the purpose of relieving the bottleneck of existing methods.

By Pu Li, Tao Tan, Hong Xie, Xiaoyu Shi, Mingsheng Shang

reinforcement-learning benchmarks safety

arXiv AI

4d ago

V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents

arXiv:2609.37250v1 Announce Type: cross Abstract: World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrai...

By Yang Zhang, Jiangyuan Zhao, Chenyou Fan, Jiayu Hu, Xiu Yuan, Chenjia Bai, Xiu Li

diffusion multimodal

Related stories