The paper introduces an operator-theoretic approach to offline reinforcement learning for continuous-time stochastic systems. By linking reinforcement learning to the Hamilton‑Jacobi‑Bellman equation, it proposes a dynamic programming recursion that represents the world model through the infinitesimal generator of controlled diffusion processes learned in a reproducing kernel Hilbert space. The authors establish global convergence of the value function under small generator‑approximation error and provide explicit finite‑sample guarantees tied to system properties such as smoothness and stability.
By Nicolas Hoischen, Petar Bevanda, Max Beier, Stefan Sosnowski, Boris Houska, Sandra Hirche
The paper presents time‑uniform self‑normalized concentration bounds for stochastic processes in Hilbert spaces with vector‑valued noise, enabling regression‑error guarantees for both linear and nonlinear parametric operators. These results apply to possibly infinite‑dimensional inputs and outputs without requiring independence or mixing assumptions, and are derived in the context of sequentially collected, dependent data such as adaptive experimental design and dynamical‑system modelling.
By Rafael Oliveira
arXiv:2607. 22399v1 Announce Type: cross Abstract: We consider the problem of learning from a single finite trajectory of an ergodic stochastic dynamical system.
By Oleksii Kachaiev, Silvia Villa, Lorenzo Rosasco
arXiv:2402.04691v5 Announce Type: replace-cross
Abstract: This study investigates the use of stochastic gradient descent (SGD) to learn operators between general Hilbert spaces. We study weak and str...
By Lei Shi, Jia-Qi Yang
The paper introduces a continuous‑time policy gradient framework that works with discrete observation data. It derives performance difference lemmas and policy gradient formulas for randomized and parametrized policies, proving exponential convergence for an entropy‑regularized policy flow and providing explicit convergence bounds for discrete updates. By integrating with PhiBE, the authors create model‑free actor‑critic algorithms that maintain the continuous‑time structure, and demonstrate their effectiveness on both linear‑quadratic and nonlinear robotic control tasks.
By Martin Hernandez, Xinyan Qing, Mo Zhou, Yuhua Zhu
The paper investigates continuous‑time stochastic control problems with unknown drift and running reward functions, using an exploratory reinforcement learning framework that incorporates relaxed controls and entropy regularization. It develops policy‑iteration algorithms based on probabilistic representations of the optimal value function and its gradient, proving convergence and demonstrating performance through numerical examples. The study also extends to a special case with control‑dependent diffusion, requiring a Hessian representation.
By Jin Ma, Gaozhan Wang, Jianfeng Zhang, Xunyu Zhou