arXiv AI

Introduction to optimization methods for training SciML models

arXiv:2601. 10222v2 Announce Type: replace-cross Abstract: Optimization is central to both modern machine learning (ML) and scientific machine learning (SciML), yet the structure of the underlying optimization problems differs substantially across these domains.

arXiv Machine Learning

Jul 27

Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers

arXiv:2607. 22004v1 Announce Type: new Abstract: Energy natural gradient descent (ENGD) aligns parameter updates with the curvature of an underlying function-space energy, but existing formulations assume an unconstrained Euclidean parameter domain.

By Zhangyong Liang, Huanhuan Gao

benchmarks

arXiv Machine Learning

Jun 10

Accelerating SAV-based optimization via randomized low-rank Hessian approximation

arXiv:2606. 10562v1 Announce Type: cross Abstract: We propose a new optimization method, the Nystr\"om-enhanced relaxed scalar auxiliary variable method (N-RSAV), which incorporates curvature information into the RSAV framework to accelerate convergence while preserving an unconditional modified energy dissipation law.

By Ryo Sagawa, Daisuke Furihata, Yuto Miyatake

arXiv AI

Jul 24

PILD: Physics-Informed Learning via Diffusion

arXiv:2601. 21284v2 Announce Type: replace-cross Abstract: Diffusion models have emerged as powerful generative tools for modeling complex data distributions, yet their purely data-driven nature limits applicability in engineering and scientific problems where physical laws must be respected.

By Tianyi Zeng, Tianyi Wang, Jiaru Zhang, Zimo Zeng, Feiyang Zhang, Yiming Xu, Sikai Chen, Junfeng Jiao, Christian Claudel, Xinbo Chen

diffusion safety

arXiv Machine Learning

Jul 27

Latent PDE mapping for efficient physics-informed learning across geometries with limited data

arXiv:2607. 22215v1 Announce Type: new Abstract: In this study, we introduce latent PDE mapping, a broadly applicable physics-informed learning technique designed to enable efficient geometric generalization with sparse training data.

By Ingvild Askim Adde, Mary M. Maleckar, Gabriel Balaban

benchmarks

arXiv Machine Learning

Jul 7

Interpretability and Generalization Bounds for Learning Spatial Physics

arXiv:2506. 15199v4 Announce Type: replace Abstract: While there are many applications of ML to scientific problems that look promising, visuals can be deceiving.

By Alejandro Francisco Queiruga, Theo Gutman-Solo, Shuai Jiang

benchmarks safety

arXiv Machine Learning

Jul 16

Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning

arXiv:2605. 07914v2 Announce Type: replace Abstract: Sharpness-aware and gradient-alignment methods have been shown to improve generalization, however each family of methods targets a single geometric property of the loss landscape, while ignoring the other.

By Aristotelis Ballas, Christos Diou

benchmarks safety

arXiv Machine Learning

Jul 7

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

arXiv:2406. 14340v2 Announce Type: replace-cross Abstract: The standard stochastic gradient descent (SGD) optimization method, as well as adaptive methods such as the Adam optimizer fail to converge if the learning rates do not converge to zero (particularly, in the situation of constant learning rates).

By Steffen Dereich, Arnulf Jentzen, Adrian Riekert

arXiv Machine Learning

Jun 18

BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training

arXiv:2606. 18650v1 Announce Type: new Abstract: As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive learning trajectories.

By Jiaxing Wang, Deping Xiang, Jin Xu, Zirui Liu, Zicheng Zhang, Guoqiang Gong, Jun Fang, Chao Liu, Pengzhang Liu, Tongxuan Liu, Ke Zhang, Qixia Jiang

llms benchmarks

arXiv Machine Learning

Jun 25

A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficients

arXiv:2606. 24999v1 Announce Type: new Abstract: High-dimensional partial differential equations (PDEs) with unknown coefficients arise widely in scientific machine learning, including continuous-time reinforcement learning, yet solving them efficiently in a data-driven way remains challenging.

By Yanwei Jia, Du Ouyang, Huy\^en Pham, Xun Yu Zhou

reinforcement-learning safety

arXiv Machine Learning

Jun 19

Learning universal approximations for partial differential equations with Physics-Informed Broad Learning System

arXiv:2606. 19754v1 Announce Type: new Abstract: Partial differential equations (PDEs) play a central role in modeling complex physical, biological, and engineering systems.

By Zhiwen Yu, Derong Yang, Liujian Zhang, Kaixiang Yang, Peilin Zhan, Jianmin Lv, Jane You, C. L. Philip Chen

arXiv Machine Learning

Jun 9

Learning to Optimize by Differentiable Programming

arXiv:2601. 16510v3 Announce Type: replace-cross Abstract: Solving massive-scale optimization problems requires scalable first-order methods with low per-iteration cost.

By Liping Tao, Xindi Tong, Chee Wei Tan

rag

arXiv Machine Learning

Jul 30

Adaptive Gradient-Based Methods for a Broader Class of Optimization Problems under Performative Prediction

arXiv:2607. 26562v1 Announce Type: cross Abstract: We study optimization under performative prediction, where deploying a model affects the future data distribution.

By Hiroki Hamaguchi, Yuya Hikima, Hiroshi Sawada, Akiko Takeda

Related stories