arXiv Machine Learning
Reflective Policy Optimization
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 22
Transductive Off-policy Proximal Policy Optimization
arXiv:2406.03894v2 Announce Type: replace Abstract: Proximal Policy Optimization (PPO) is a popular model-free reinforcement learning algorithm, esteemed for its simplicity and efficacy. However, due...
arXiv Machine Learning
Oct 3
Reusing Past Samples in Proximal Policy Optimization: When and How Does It Help?
The paper investigates how reusing past samples can improve the sample efficiency of Proximal Policy Optimization (PPO). Two variants, wPPO-U and wPPO-BH, are introduced within a multiple importance weighting framework, each reusing data from recent iterations while preserving core PPO mechanics. The authors derive theoretical policy improvement bounds for both variants and empirically evaluate their impact on continuous control tasks.
arXiv Machine Learning
Aug 14
Variance Reduction Based Experience Replay for Policy Optimization
arXiv:2602. 05379v2 Announce Type: replace-cross Abstract: Effective reinforcement learning (RL) for complex stochastic systems requires leveraging historical data to improve sample efficiency and accelerate policy optimization.
arXiv Machine Learning
Jun 19
Approximate Next Policy Sampling: Replacing Conservative Target Policy Updates in Deep RL
arXiv:2605. 05481v2 Announce Type: replace Abstract: We revisit a classic "chicken-and-egg" problem in reinforcement learning: to safely improve a policy, the value function must be accurate on the state-visitation distribution of the updated policy.
OpenAI Blog
Jul 20, 2017
Proximal Policy Optimization
We’re releasing a new class of reinforcement learning algorithms, Proximal Policy Optimization (PPO), which perform comparably or better than state-of-the-art approaches while being much simpler to implement and tune. PPO has become the default reinforcement learning algorithm at OpenAI because of its ease of use and good performance.
arXiv AI
Oct 12
Beyond State-Wise Mirror Descent: Offline Policy Optimization with Parametric Policies
The paper extends theoretical guarantees for offline reinforcement learning to parameterized policy classes over large or continuous action spaces, overcoming limitations of prior algorithms that required finite action sets and state‑wise mirror descent. By addressing contextual coupling and linking mirror descent to natural policy gradients, the authors provide new analyses, algorithmic insights, and a surprising connection between offline RL and imitation learning.