arXiv Machine Learning July 21, 2026 By Zhangyong Liang, Huanhuan Gao

Stochastic Dimension Zeroth-Order Estimator: Stable and Memory-Efficient Training of PINNs

Read the original on arXiv Machine Learning →

arXiv:2603. 24002v3 Announce Type: replace Abstract: Physics-Informed Neural Networks (PINNs) for high-dimensional and high-order partial differential equations (PDEs) are primarily constrained by the $\mathcal{O}(d^k)$ spatial derivative complexity and the $\mathcal{O}(P)$ memory overhead of backpropagation (BP).

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 9

Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNs

arXiv:2603. 29237v2 Announce Type: replace Abstract: Enforcing prescribed global integral constraints in mesh-free neural PDE solvers is challenging in high-dimensional domains.

By Zhangyong Liang, Huanhuan Gao

arXiv Machine Learning

Jun 3

Scalable Derivative Gaussian Processes via Exact Gradient Reduction

arXiv:2606. 02909v1 Announce Type: cross Abstract: Gradient observations can substantially improve Gaussian process (GP) surrogates, particularly in high-dimensional settings where function evaluations are expensive.

By Hyunseok Seung, Matthias Katzfuss

benchmarks

arXiv Machine Learning

Jun 16

SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums

arXiv:2606. 15832v1 Announce Type: new Abstract: Empirical risk minimization on massive datasets naturally exhibits a nested double finite-sum structure, where $N=nm$ total samples are logically or physically partitioned into $n$ blocks of size $m$ (e.

By Igor Sokolov, Laurent Condat, Peter Richt\'arik

efficiency benchmarks

arXiv Machine Learning

Jun 2

Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise

arXiv:2605. 18528v2 Announce Type: replace-cross Abstract: A growing lesson from neural network optimization is that optimizer design should respect how the model is parametrized.

By Jiayu Zhang, Tianyi Lin

arXiv Machine Learning

Jun 5

Stochastic-Dimension Frozen Sampled Neural Network for High-Dimensional Gross-Pitaevskii Equations on Unbounded Domains

arXiv:2604. 09361v3 Announce Type: replace Abstract: This paper introduces the Stochastic-Dimension Frozen Sampled Neural Network (SD-FSNN), a novel computational framework for solving high-dimensional Gross-Pitaevskii equation (GPE) on unbounded domain.

By Zhangyong Liang, Tingfeng Wang, Xiaofei Zhao

benchmarks

arXiv Machine Learning

Jun 10

Accelerating SAV-based optimization via randomized low-rank Hessian approximation

arXiv:2606. 10562v1 Announce Type: cross Abstract: We propose a new optimization method, the Nystr\"om-enhanced relaxed scalar auxiliary variable method (N-RSAV), which incorporates curvature information into the RSAV framework to accelerate convergence while preserving an unconditional modified energy dissipation law.

By Ryo Sagawa, Daisuke Furihata, Yuto Miyatake

Related stories

Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNs

Scalable Derivative Gaussian Processes via Exact Gradient Reduction

SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums

Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise

Stochastic-Dimension Frozen Sampled Neural Network for High-Dimensional Gross-Pitaevskii Equations on Unbounded Domains

Accelerating SAV-based optimization via randomized low-rank Hessian approximation