← Back to all news
arXiv Machine Learning By Martin Hernandez, Xinyan Qing, Mo Zhou, Yuhua Zhu

Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free Algorithms

Read the original on arXiv Machine Learning →

The paper introduces a continuous‑time policy gradient framework that works with discrete observation data. It derives performance difference lemmas and policy gradient formulas for randomized and parametrized policies, proving exponential convergence for an entropy‑regularized policy flow and providing explicit convergence bounds for discrete updates. By integrating with PhiBE, the authors create model‑free actor‑critic algorithms that maintain the continuous‑time structure, and demonstrate their effectiveness on both linear‑quadratic and nonlinear robotic control tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 15

Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

The paper investigates continuous‑time stochastic control problems with unknown drift and running reward functions, using an exploratory reinforcement learning framework that incorporates relaxed controls and entropy regularization. It develops policy‑iteration algorithms based on probabilistic representations of the optimal value function and its gradient, proving convergence and demonstrating performance through numerical examples. The study also extends to a special case with control‑dependent diffusion, requiring a Hessian representation.

By Jin Ma, Gaozhan Wang, Jianfeng Zhang, Xunyu Zhou
diffusionreinforcement-learning
More like this →
arXiv Machine Learning
Oct 12

Infinitesimal Operator World Models: Finite-Sample Guarantees for Offline Reinforcement Learning

The paper introduces an operator-theoretic approach to offline reinforcement learning for continuous-time stochastic systems. By linking reinforcement learning to the Hamilton‑Jacobi‑Bellman equation, it proposes a dynamic programming recursion that represents the world model through the infinitesimal generator of controlled diffusion processes learned in a reproducing kernel Hilbert space. The authors establish global convergence of the value function under small generator‑approximation error and provide explicit finite‑sample guarantees tied to system properties such as smoothness and stability.

By Nicolas Hoischen, Petar Bevanda, Max Beier, Stefan Sosnowski, Boris Houska, Sandra Hirche
agentsdiffusionreinforcement-learningrobotics
More like this →