← Back to all news
arXiv Machine Learning By Nicolas Hoischen, Petar Bevanda, Max Beier, Stefan Sosnowski, Boris Houska, Sandra Hirche

Infinitesimal Operator World Models: Finite-Sample Guarantees for Offline Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces an operator-theoretic approach to offline reinforcement learning for continuous-time stochastic systems. By linking reinforcement learning to the Hamilton‑Jacobi‑Bellman equation, it proposes a dynamic programming recursion that represents the world model through the infinitesimal generator of controlled diffusion processes learned in a reproducing kernel Hilbert space. The authors establish global convergence of the value function under small generator‑approximation error and provide explicit finite‑sample guarantees tied to system properties such as smoothness and stability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
1d ago

Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free Algorithms

The paper introduces a continuous‑time policy gradient framework that works with discrete observation data. It derives performance difference lemmas and policy gradient formulas for randomized and parametrized policies, proving exponential convergence for an entropy‑regularized policy flow and providing explicit convergence bounds for discrete updates. By integrating with PhiBE, the authors create model‑free actor‑critic algorithms that maintain the continuous‑time structure, and demonstrate their effectiveness on both linear‑quadratic and nonlinear robotic control tasks.

By Martin Hernandez, Xinyan Qing, Mo Zhou, Yuhua Zhu
reinforcement-learningrobotics
More like this →
arXiv Machine Learning
Oct 12

Data-Driven Stochastic Optimal Control in Reproducing Kernel Hilbert Spaces

The paper presents a fully data-driven method for optimal control of nonlinear control‑affine systems modeled as stochastic diffusions. It embeds state probability densities into a reproducing kernel Hilbert space to learn Markov transition operators, which are then used in convex operator‑theoretic Hamilton‑Jacobi‑Bellman recursions that scale linearly with state dimensionality. Numerical experiments show the approach can solve diverse nonlinear control tasks, such as depth regulation of an autonomous underwater vehicle.

By Nicolas Hoischen, Petar Bevanda, Stefan Sosnowski, Sandra Hirche, Boris Houska
agentsdiffusion
More like this →
arXiv Machine Learning
Sep 15

Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

The paper investigates continuous‑time stochastic control problems with unknown drift and running reward functions, using an exploratory reinforcement learning framework that incorporates relaxed controls and entropy regularization. It develops policy‑iteration algorithms based on probabilistic representations of the optimal value function and its gradient, proving convergence and demonstrating performance through numerical examples. The study also extends to a special case with control‑dependent diffusion, requiring a Hessian representation.

By Jin Ma, Gaozhan Wang, Jianfeng Zhang, Xunyu Zhou
diffusionreinforcement-learning
More like this →