The paper introduces two new streaming deep reinforcement learning algorithms—Streaming Soft Actor-Critic (S2AC) and Streaming Deterministic Actor-Critic (SDAC)—designed to work with state‑of‑the‑art batch RL methods and suitable for on‑device fine‑tuning, such as Sim2Real transfer. These algorithms achieve performance comparable to existing streaming baselines on standard benchmarks without the need for extensive per‑environment hyperparameter tuning. The authors also examine the challenges of transitioning from batch to streaming RL, demonstrating that a naive approach can degrade pre‑trained policy performance and propose a principled solution to mitigate this issue.
arXiv:2410.14606v3 Announce Type: replace Abstract: Learning from a stream of experience as it arrives, also known as streaming learning, is a core part of natural learning. However, reliable streami...
arXiv:2503. 03660v4 Announce Type: replace Abstract: We introduce a sequence-conditioned critic for Soft Actor-Critic (SAC) that models trajectory context with a lightweight Transformer and trains on aggregated $N$-step targets.
arXiv:2606. 11087v1 Announce Type: cross Abstract: Expressive continuous control policies, such as diffusion and flow models, form the backbone of recent advances in scaling imitation learning for simulated and real robot control.
arXiv:2605. 05481v2 Announce Type: replace Abstract: We revisit a classic "chicken-and-egg" problem in reinforcement learning: to safely improve a policy, the value function must be accurate on the state-visitation distribution of the updated policy.
arXiv:2107. 08183v2 Announce Type: replace Abstract: High-dimensional state and action spaces combined with sparse reward structures in reinforcement learning (RL) environments typically require advanced control architectures.
arXiv:2607. 26509v1 Announce Type: new Abstract: Deep off-policy reinforcement learning algorithms for continuous control typically rely on neural value function approximation to guide policy improvement.
arXiv:2602. 05999v3 Announce Type: replace Abstract: How does the amount of compute available to a reinforcement learning (RL) policy affect its learning?
arXiv:2608.20909v1 Announce Type: new Abstract: Offline RL methods commonly jointly train the actor and critic, where the critic is used to guide the actor toward higher-value actions. This coupled l...
The paper introduces LAC (Light Actor, deep Critic), an offline reinforcement learning approach that allocates model capacity to a deep critic rather than a complex actor to improve inference efficiency. It addresses three failure modes—optimization, bootstrap-noise amplification, and value-range drift—using a residual MLP backbone, n‑step bootstrap targets, and a categorical cross‑entropy loss. Experiments on OGBench show LAC matches state‑of‑the‑art diffusion and flow‑matching baselines while reducing inference latency by up to four times.
arXiv:2601. 23075v2 Announce Type: replace Abstract: On-policy Reinforcement Learning (RL) remains a dominant paradigm for continuous control, yet standard implementations rely on Gaussian actors and relatively shallow MLP policies, often leading to brittle optimization when gradients are noisy, and policy updates must be conservative.
arXiv:2605. 24709v2 Announce Type: replace Abstract: Streaming reinforcement learning has emerged as an online learning paradigm that conforms to the restrictions of natural learning agents that process data incrementally, i.
Massively parallel simulation changes the data regime in which off-policy reinforcement learning (RL) is trained, challenging stabilizers designed for data-limited replay. Through controlled experimen...