Policy Gradient with PyTorch

Variance reduction for policy gradient with action-dependent factorized baselines

Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement Learning

arXiv:2609.06882v1 Announce Type: cross Abstract: Diffusion policies offer a powerful and expressive parameterization for continuous control. Yet, their integration with reinforcement learning remain...

Randomized Advantage Transformation (RAT): Computing Natural Policy Gradients via Direct Backpropagation

arXiv:2605. 18591v2 Announce Type: replace Abstract: Natural policy gradients improve optimization by accounting for the geometry of distribution space, but their practical use is limited by the cost of estimating and inverting the Fisher matrix.

Equivalence between policy gradients and soft Q-learning

Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

The paper investigates continuous‑time stochastic control problems with unknown drift and running reward functions, using an exploratory reinforcement learning framework that incorporates relaxed controls and entropy regularization. It develops policy‑iteration algorithms based on probabilistic representations of the optimal value function and its gradient, proving convergence and demonstrating performance through numerical examples. The study also extends to a special case with control‑dependent diffusion, requiring a Hessian representation.

Second-Order Actor-Critic Methods for Discounted MDPs via Policy Hessian Decomposition

arXiv:2605. 14982v2 Announce Type: replace-cross Abstract: We address the discounted reward setting in reinforcement learning (RL).