arXiv:2510. 13704v2 Announce Type: replace-cross Abstract: Recent works have proposed accelerating the wall-clock training time of actor-critic methods via the use of large-scale environment parallelization; unfortunately, these can sometimes still require large number of environment interactions to achieve a desired level of performance.
arXiv:2606. 05555v1 Announce Type: new Abstract: Scaling reinforcement learning (RL) to diverse multitask settings remains a central challenge.
Scaling reinforcement learning (RL) to diverse multitask settings remains a central challenge. While recent advances in model-based RL achieve strong performance, they rely on planning and complex training pipelines, making it unclear which components are essential for scalability.
arXiv:2601. 23075v2 Announce Type: replace Abstract: On-policy Reinforcement Learning (RL) remains a dominant paradigm for continuous control, yet standard implementations rely on Gaussian actors and relatively shallow MLP policies, often leading to brittle optimization when gradients are noisy, and policy updates must be conservative.
arXiv:2602. 12643v2 Announce Type: replace-cross Abstract: We present Unified Latent Dynamics (ULD), a novel reinforcement learning algorithm that unifies the efficiency of model-free methods with the representational strengths of model-based approaches, without incurring planning overhead.
arXiv:2608. 07870v1 Announce Type: new Abstract: Improving sample efficiency remains a core challenge in reinforcement learning (RL), especially in real-world settings like robotics, where data collection is costly.
arXiv:2608. 16798v1 Announce Type: cross Abstract: Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment.
arXiv:2410.14606v3 Announce Type: replace Abstract: Learning from a stream of experience as it arrives, also known as streaming learning, is a core part of natural learning. However, reliable streami...
arXiv:2503. 03660v4 Announce Type: replace Abstract: We introduce a sequence-conditioned critic for Soft Actor-Critic (SAC) that models trajectory context with a lightweight Transformer and trains on aggregated $N$-step targets.
arXiv:2605. 26012v2 Announce Type: replace-cross Abstract: Deep reinforcement learning (RL) agents commonly rely on high-dimensional neural representations, despite growing evidence that task-relevant value and policy structure may be intrinsically low-dimensional.
The paper introduces two new streaming deep reinforcement learning algorithms—Streaming Soft Actor-Critic (S2AC) and Streaming Deterministic Actor-Critic (SDAC)—designed to work with state‑of‑the‑art batch RL methods and suitable for on‑device fine‑tuning, such as Sim2Real transfer. These algorithms achieve performance comparable to existing streaming baselines on standard benchmarks without the need for extensive per‑environment hyperparameter tuning. The authors also examine the challenges of transitioning from batch to streaming RL, demonstrating that a naive approach can degrade pre‑trained policy performance and propose a principled solution to mitigate this issue.
The paper introduces FastRL, a reinforcement learning framework designed to enhance the efficiency of Group Relative Policy Optimization (GRPO) and its variants. FastRL employs an advantage-aware pruning strategy that retains high-advantage trajectories while preserving gradient diversity, and an adaptive rollout sampling mechanism that adjusts sampling scale during training based on historical pruning data. Experiments show that FastRL can be integrated into GRPO, DAPO, and GSPO, yielding a 2.07× speedup on Geometry3K and GeoQA8K-R1V and a 1.64% accuracy improvement on visual reasoning benchmarks.
The paper proposes using Low‑Rank Adaptation (LoRA) as a structural regularizer for critic learning in off‑policy reinforcement learning. By freezing randomly initialized base matrices and optimizing only low‑rank adapters, the method constrains weight changes to low‑rank residuals, reducing critic loss and improving policy performance. Experiments on state‑based continuous‑control tasks with SAC, FastTD3, and their SimbaV2 variants show that LoRA achieves the best or competitive results on most tasks.