← Back to all news
arXiv Machine Learning

Continuous Delayed-Memory Stochastic Gradient Descent and Continuous-Time Reinforcement Learning from History of Astrophysical Time Series Studies

The paper introduces a new optimization method called Continuous‑Delayed‑Memory Stochastic Gradient Descent, which incorporates past states of the iteration process to improve exploration and convergence compared to vanilla SGD. It also proposes a continuous‑time reinforcement learning framework that uses policy gradients for exploratory policies, showing that its optimality conditions recover the Gibbs policy from earlier studies. The work builds on stochastic differential equations adapted with neural network parameterizations to model quasars’ stochastic brightness variations from light‑curve data.

Related stories

arXiv Machine Learning
Sep 15

Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

The paper investigates continuous‑time stochastic control problems with unknown drift and running reward functions, using an exploratory reinforcement learning framework that incorporates relaxed controls and entropy regularization. It develops policy‑iteration algorithms based on probabilistic representations of the optimal value function and its gradient, proving convergence and demonstrating performance through numerical examples. The study also extends to a special case with control‑dependent diffusion, requiring a Hessian representation.

By Jin Ma, Gaozhan Wang, Jianfeng Zhang, Xunyu Zhou
diffusionreinforcement-learning
More like this →
arXiv Machine Learning
Oct 15

Beyond Time Induction: Structured Conditional Neural Solvers for Parametric Stochastic-Volatility Pricing

The paper introduces BeTINN, a structured conditional neural network architecture that extends time‑induced modulation with multi‑directional time‑parameter conditioning and a joint residual pathway. BeTINN is evaluated on the Heston stochastic‑volatility PDE, showing improved accuracy for both option values and key sensitivity metrics compared to baseline models. Ablation studies highlight the contributions of each architectural component, and the authors observe that physics‑based validation loss does not fully capture derivative fidelity, underscoring the need for derivative‑aware evaluation when sensitivities are important.

By Jose Miguel Lara Rangel, Mixtli Vazquez Mendoza
More like this →
arXiv Machine Learning
Jul 7

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

arXiv:2406. 14340v2 Announce Type: replace-cross Abstract: The standard stochastic gradient descent (SGD) optimization method, as well as adaptive methods such as the Adam optimizer fail to converge if the learning rates do not converge to zero (particularly, in the situation of constant learning rates).

By Steffen Dereich, Arnulf Jentzen, Adrian Riekert
More like this →
arXiv Machine Learning
6d ago

A Model-Free Continuous-Time RL Algorithm from Discrete-Time Off-Policy Data with End-to-End Guarantees

The paper introduces a model‑free continuous‑time reinforcement learning algorithm that learns from a fixed batch of discrete‑time off‑policy data. It builds on the PhiBE‑Q framework to create a PhiBE‑PI policy iteration that uses a single state‑action critic, avoiding explicit drift or diffusion estimation. The authors provide a local end‑to‑end suboptimality guarantee under linear function approximation, analyze error propagation from discretization, function approximation, and finite samples, and reveal a bias‑variance trade‑off where finer time discretization can worsen statistical error.

By Chaorui Yao, Yuming Paul Zhang, Lin Yang, Yuhua Zhu
diffusionreinforcement-learningsafety
More like this →
arXiv Machine Learning
Jun 25

A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficients

arXiv:2606. 24999v1 Announce Type: new Abstract: High-dimensional partial differential equations (PDEs) with unknown coefficients arise widely in scientific machine learning, including continuous-time reinforcement learning, yet solving them efficiently in a data-driven way remains challenging.

By Yanwei Jia, Du Ouyang, Huy\^en Pham, Xun Yu Zhou
reinforcement-learningsafety
More like this →
arXiv Statistics ML
Oct 1

Learning to Plan from Random Exploration

arXiv:2609.38383v1 Announce Type: cross Abstract: Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without po...

By Deqian Kong, Guangyan Sun, Sheng Cheng, Sirui Xie, Bo Pang, Jianwen Xie, Tony Geng, Caiwen Ding, Ying Nian Wu
ragagentsdiffusionrobotics
More like this →