arXiv Machine Learning August 28, 2026 By Guhyeon Kang, Jaehwi Lee, Minhae Kwon

Simple Actors and Deep Critics for Scalable Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces LAC (Light Actor, deep Critic), an offline reinforcement learning approach that allocates model capacity to a deep critic rather than a complex actor to improve inference efficiency. It addresses three failure modes—optimization, bootstrap-noise amplification, and value-range drift—using a residual MLP backbone, n‑step bootstrap targets, and a categorical cross‑entropy loss. Experiments on OGBench show LAC matches state‑of‑the‑art diffusion and flow‑matching baselines while reducing inference latency by up to four times.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 11

SMAC: Score-Matched Actor-Critics for Robust Offline-to-Online Transfer

arXiv:2602. 17632v3 Announce Type: replace-cross Abstract: Modern offline Reinforcement Learning (RL) methods find performant actor-critics, however, fine-tuning these actor-critics online with value-based RL algorithms typically causes immediate drops in performance.

By Nathan Samuel de Lara, Florian Shkurti

reinforcement-learning fine-tuning

arXiv Machine Learning

Aug 24

Decoupling Policy Extraction for Offline Reinforcement Learning

arXiv:2608.20909v1 Announce Type: new Abstract: Offline RL methods commonly jointly train the actor and critic, where the critic is used to guide the actor toward higher-value actions. This coupled l...

By Xuyao Lin, Yixiang Shan, Jinru Duan, Tao Yang, Xinyu Zhao, Runyu Lei, Yiming Zhao, Jiaxin Fan, Zongbao Feng, Peng Jia

rag reinforcement-learning

arXiv Machine Learning

Aug 4

ReBRAC-v2: The Return of the King

arXiv:2608. 01205v1 Announce Type: new Abstract: Recent offline reinforcement learning methods increasingly rely on expressive generative policies and specialized value-guidance mechanisms.

By Denis Tarasov, Robert K. Katzschmann

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

RN-D: Discretized Categorical Actors for On-Policy Reinforcement Learning

arXiv:2601. 23075v2 Announce Type: replace Abstract: On-policy Reinforcement Learning (RL) remains a dominant paradigm for continuous control, yet standard implementations rely on Gaussian actors and relatively shallow MLP policies, often leading to brittle optimization when gradients are noisy, and policy updates must be conservative.

By Yuexin Bian, Jie Feng, Tao Wang, Yijiang Li, Sicun Gao, Yuanyuan Shi

reinforcement-learning benchmarks

arXiv AI

Jun 16

QPILOTS: Efficient Test-Time Q-Steering for Flow Policies

arXiv:2606. 14801v1 Announce Type: cross Abstract: Flow-matching and diffusion policies are expressive action generators, but optimizing them with temporal-difference reinforcement learning (RL) remains difficult.

By Yifan Ruan, Chenyang Cao, Andreas Burger, Ali Pesaranghader, Kaveh Kamali, Jaehong Kim, Nandita Vijaykumar, Alan Aspuru-Guzik, Igor Gilitschenski, Nicholas Rhinehart

diffusion reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 30

Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning

arXiv:2607. 26509v1 Announce Type: new Abstract: Deep off-policy reinforcement learning algorithms for continuous control typically rely on neural value function approximation to guide policy improvement.

By Gong Gao, Xiao Lai, Ziqi Xie, Guojie Chen, Xianhui Liu, Weidong Zhao

reinforcement-learning safety

One email a morning, machine-written

Related stories

SMAC: Score-Matched Actor-Critics for Robust Offline-to-Online Transfer

Decoupling Policy Extraction for Offline Reinforcement Learning

ReBRAC-v2: The Return of the King

RN-D: Discretized Categorical Actors for On-Policy Reinforcement Learning

QPILOTS: Efficient Test-Time Q-Steering for Flow Policies

Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning