The paper introduces a continuous‑time policy gradient framework that works with discrete observation data. It derives performance difference lemmas and policy gradient formulas for randomized and parametrized policies, proving exponential convergence for an entropy‑regularized policy flow and providing explicit convergence bounds for discrete updates. By integrating with PhiBE, the authors create model‑free actor‑critic algorithms that maintain the continuous‑time structure, and demonstrate their effectiveness on both linear‑quadratic and nonlinear robotic control tasks.
By Martin Hernandez, Xinyan Qing, Mo Zhou, Yuhua Zhu
arXiv:2607. 06935v1 Announce Type: cross Abstract: Reinforcement learning (RL) is increasingly grounded in tools from probability, optimization, and operator theory.
By Denis Belomestny, Alexander Gasnikov, Egor Gladin, Alexey Naumov, Artemy Rubtsov, Yuri Sapronov, Daniil Tiapkin, Nikita Yudin
arXiv:2510. 03494v2 Announce Type: replace Abstract: We study finite-horizon offline reinforcement learning (RL) with function approximation for both policy evaluation and policy optimization.
By Volodymyr Tkachuk, Csaba Szepesv\'ari, Xiaoqi Tan
The paper presents a fully data-driven method for optimal control of nonlinear control‑affine systems modeled as stochastic diffusions. It embeds state probability densities into a reproducing kernel Hilbert space to learn Markov transition operators, which are then used in convex operator‑theoretic Hamilton‑Jacobi‑Bellman recursions that scale linearly with state dimensionality. Numerical experiments show the approach can solve diverse nonlinear control tasks, such as depth regulation of an autonomous underwater vehicle.
By Nicolas Hoischen, Petar Bevanda, Stefan Sosnowski, Sandra Hirche, Boris Houska
The paper investigates continuous‑time stochastic control problems with unknown drift and running reward functions, using an exploratory reinforcement learning framework that incorporates relaxed controls and entropy regularization. It develops policy‑iteration algorithms based on probabilistic representations of the optimal value function and its gradient, proving convergence and demonstrating performance through numerical examples. The study also extends to a special case with control‑dependent diffusion, requiring a Hessian representation.
By Jin Ma, Gaozhan Wang, Jianfeng Zhang, Xunyu Zhou
arXiv:2610.28915v1 Announce Type: new
Abstract: In this paper, we study continuous-time reinforcement learning, where the state follows an unknown stochastic differential equation but only a fixed of...
By Chaorui Yao, Yuming Paul Zhang, Lin Yang, Yuhua Zhu
arXiv:2512. 04697v3 Announce Type: replace-cross Abstract: This paper studies the continuous-time reinforcement learning (RL) for optimal switching problems across multiple regimes.
By Yijie Huang, Mengge Li, Xiang Yu, Zhou Zhou
arXiv:2506. 08121v2 Announce Type: replace-cross Abstract: We introduce a continuous policy-value iteration algorithm where the approximations of the value function of a stochastic control problem and the optimal control are simultaneously updated through Langevin-type dynamics.
By Qi Feng, Gu Wang
The paper presents a large deviations framework for efficient data acquisition in infinite-horizon reinforcement learning, introducing the exponential decay rate of policy-selection error probability as a key efficiency metric. It derives a variational characterization leading to a nested optimization problem, then proposes a tractable convex relaxation and a lazy one-step projected subgradient method to construct an adaptive data acquisition policy. The resulting algorithm is shown to be near-robustly optimal under the proposed criterion, with extensions to linear function approximation and supporting numerical experiments.
By Mingjie Hu, Jian-Qiang Hu, Enlu Zhou
arXiv:2607. 22399v1 Announce Type: cross Abstract: We consider the problem of learning from a single finite trajectory of an ergodic stochastic dynamical system.
By Oleksii Kachaiev, Silvia Villa, Lorenzo Rosasco
The paper introduces a new Bellman operator that merges performance and safety objectives into a single joint value function for reinforcement learning. Temporal difference learning with this operator converges under a two‑timescale stochastic approximation, estimating safety on a fast timescale and joint value on a slow one. Theoretical analysis shows convergence to safety‑constrained optimal policies, and experiments on continuous control tasks confirm stable learning with near‑zero safety violations.
By Nishanth Arun Rao, Royina Karegoudra Jayanth, Benjamin Eysenbach, Jaime Fern\'andez Fisac
arXiv:2606. 16846v1 Announce Type: cross Abstract: We study the operator-theoretic core of Q-learning in continuous-time stochastic control with continuous states and actions.
By Qian Qi