OpenAI Blog
Variance reduction for policy gradient with action-dependent factorized baselines
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
arXiv Machine Learning
5d ago
Variance Reduction Based Experience Replay for Policy Optimization
arXiv:2602. 05379v2 Announce Type: replace-cross Abstract: Effective reinforcement learning (RL) for complex stochastic systems requires leveraging historical data to improve sample efficiency and accelerate policy optimization.
arXiv Machine Learning
Jul 31
Variance-Aware Baselines and Adaptive Learning Rates for Reinforcement Learning with Verifiable Rewards
arXiv:2511. 23310v3 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as an effective paradigm for post-training large language models, yet the design of its baselines and learning-rate schedules remains largely heuristic.
arXiv AI
Jun 6
Retry Policy Gradients in Continuous Action Spaces
arXiv:2606. 05888v1 Announce Type: new Abstract: Retry-based objectives such as pass@K and max@K optimize the best return obtained from multiple sampled trajectories, and recent work has shown that they can promote exploration without explicit exploration bonuses.
arXiv AI
Jun 19
Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic Methods
arXiv:2601. 22970v2 Announce Type: replace-cross Abstract: Policies learned via continuous actor-critic methods often exhibit erratic, high-frequency oscillations, making them unsuitable for physical deployment.
arXiv AI
Aug 7
Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination
arXiv:2605. 04568v3 Announce Type: replace-cross Abstract: State-of-the-art model-based Reinforcement Learning (RL) approaches either use gradient-free, population-based methods for planning, learned policy networks, or a combination of policy networks and planning.
arXiv Machine Learning
Aug 4
Randomized Advantage Transformation (RAT): Computing Natural Policy Gradients via Direct Backpropagation
arXiv:2605. 18591v2 Announce Type: replace Abstract: Natural policy gradients improve optimization by accounting for the geometry of distribution space, but their practical use is limited by the cost of estimating and inverting the Fisher matrix.