arXiv AI By Alena Kopani\v{c}\'akov\'a, Elisa Riccietti

Introduction to optimization methods for training SciML models

Read the original on arXiv AI →

arXiv:2601. 10222v2 Announce Type: replace-cross Abstract: Optimization is central to both modern machine learning (ML) and scientific machine learning (SciML), yet the structure of the underlying optimization problems differs substantially across these domains.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 28

Physics-Informed Stochastic Configuration Machine: A Backpropagation-Free Neural Network with Fast Training for Nonlinear Differential Equations

The paper introduces the Physics-Informed Stochastic Configuration Machine (PI‑SCM), a backpropagation‑free neural network designed for solving nonlinear differential equations. By analytically evaluating local Jacobians, PI‑SCM linearizes the physical loss, enabling optimal weight determination through generalized linear least squares and avoiding iterative nonlinear optimization. The authors present a progressive algorithmic suite—PI‑SC‑I, PI‑SC‑II, and PI‑SC‑III—prove their universal approximation properties, and show through experiments that PI‑SCM achieves high‑fidelity predictions and parameter identification while accelerating training by orders of magnitude compared to standard PINNs.

By Yuehao Song (School of Automation, Central South University, Changsha, China), Zhong Chen (School of Automation, Central South University, Changsha, China), Lihui Cen (School of Automation, Central South University, Changsha, China), Liang Wu (Johns Hopkins University, Baltimore, USA), Kai Zhang (State Key Laboratory of Simulation and Regulation of Water Cycle in River Basin, China Institute of Water Resources and Hydropower Research, Beijing, China)
arXiv Machine Learning
Jun 10

Accelerating SAV-based optimization via randomized low-rank Hessian approximation

arXiv:2606. 10562v1 Announce Type: cross Abstract: We propose a new optimization method, the Nystr\"om-enhanced relaxed scalar auxiliary variable method (N-RSAV), which incorporates curvature information into the RSAV framework to accelerate convergence while preserving an unconditional modified energy dissipation law.

By Ryo Sagawa, Daisuke Furihata, Yuto Miyatake
arXiv AI
Jul 24

PILD: Physics-Informed Learning via Diffusion

arXiv:2601. 21284v2 Announce Type: replace-cross Abstract: Diffusion models have emerged as powerful generative tools for modeling complex data distributions, yet their purely data-driven nature limits applicability in engineering and scientific problems where physical laws must be respected.

By Tianyi Zeng, Tianyi Wang, Jiaru Zhang, Zimo Zeng, Feiyang Zhang, Yiming Xu, Sikai Chen, Junfeng Jiao, Christian Claudel, Xinbo Chen
arXiv Machine Learning
Sep 15

Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning

The review explores how control theory, optimal transport, probabilistic inference, non‑equilibrium thermodynamics, and machine learning are interconnected through the optimization of free‑energy‑like functionals under dynamical or statistical constraints. It presents a conceptual thread linking these five fields and illustrates the ideas with applications in reinforcement learning, variational inference, and generative modeling. The article is written for readers without prior familiarity, beginning with physics principles.

By Emmy Blumenthal, Nikolas Claussen, Benjamin Eysenbach, Catherine Ji, Gautam Reddy, Colin Scheibner, Benjamin Sorkin