The paper introduces an Adaptive Hybrid Subspace Levenberg–Marquardt (HSLM) algorithm that tackles large‑scale nonlinear least‑squares problems by building a low‑dimensional subspace from gradient, memory, Krylov‑subspace, and randomized curvature data. It employs a deterministic adequacy monitor to adaptively enrich the subspace and decouples step acceptance from damping adjustment, using Armijo backtracking for step length and a ratio of actual to predicted reduction for damping updates. The authors prove global convergence to stationarity and local linear and superlinear convergence, and demonstrate that HSLM matches the convergence of classical and Krylov‑subspace LM while significantly reducing per‑iteration cost, especially as the parameter dimension increases.
By M. Duc Hoang, Timothy J. Lewis
arXiv:2607. 02194v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) have emerged as a promising route to solve partial differential equations, yet they have struggled to reach the precision of classical solvers.
By Joseph Webb, Sadok Jerad, Coralia Cartis
arXiv:2609. 30501v1 Announce Type: new Abstract: Although bilevel optimization (BLO) has emerged as a powerful framework for addressing many complex and nested machine learning problems in recent years, most existing studies are confined to the lower-level strongly convex (LLSC) or lower-level generally convex (LLGC) settings (i.
By Zhiyao Zhang, Menglu Yu, Alvaro Velasquez, Nathaniel D. Bastian, Jia Liu
arXiv:2606. 01122v1 Announce Type: new Abstract: We propose a five-step diagnostic protocol for residual-trained neural HJB-PIDE solvers with control-dependent L\'evy jumps, targeting a general failure mode of neural PDE methods: a learned solution can match headline scalar diagnostics while miscomputing an operator inside its training loss.
By R. Drissi
arXiv:2608. 05892v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) often fail to accurately resolve partial differential equations (PDEs) with high-frequency or multi-scale solutions, as well as strongly nonlinear problems.
By Yulun Wu, Matthieu Barreau, Miguel Aguiar, Karl H. Johansson
arXiv:2609.00789v1 Announce Type: new
Abstract: The Levenberg-Marquardt (LM) algorithm is a well-known second-order method for rapid convergence and strong robustness when training small- to medium-s...
By M. Duc Hoang
arXiv:2606. 18175v1 Announce Type: cross Abstract: We present a numerical method for the forward solution of nonlinear partial differential equations (PDEs) in which Bellman-Kalaba quasilinearization reduces the nonlinear problem to a sequence of linear subproblems, each discretized by collocation onto a trial space that is linear in its parameters and solved by a single direct linear least-squares QR factorization.
By Gbenga T. Awojinrin, Abdul-Akeem Olawoyin, Rami M. Younis
arXiv:2602.08515v3 Announce Type: replace-cross
Abstract: This work investigates shallow physics-informed neural networks (PINNs) for solving forward and inverse problems governed by nonlinear partia...
By Muhammad Luthfi Shahab, Imam Mukhlash, Hadi Susanto
arXiv:2606. 30328v1 Announce Type: cross Abstract: Rapid prototyping of algorithms is a critical step in modern machine learning.
By Disha Hegde, Jon Cockayne, Chris. J. Oates
The paper introduces DA‑EGO, an efficient global optimization algorithm that dynamically aggregates high‑dimensional design spaces into low‑dimensional subspaces for surrogate‑based search. The algorithm updates subspace variables each iteration using variable‑interaction analyses, perturbation, and ANOVA, and adaptively adjusts search ranges based on previous results. Tests on 21 benchmark functions and real turbomachinery problems demonstrate DA‑EGO’s effectiveness, especially on separable and partially separable problems, while noting case‑dependent performance on non‑separable functions.
By Qineng Wang, Zhendong Guo, Yun Chen, Guangjian Ma, Liming Song, Jun Li
arXiv:2606. 31390v1 Announce Type: cross Abstract: Low-rank matrix optimization is often carried out via the Burer-Monteiro (BM) formulation, but choosing the factorization rank $r$ is delicate and can substantially slow optimization.
By Yudong Wei, Liang Zhang, Bingcong Li, Niao He
arXiv:2609.36165v1 Announce Type: cross
Abstract: In this work, we develop a second-order optimization framework for physics-informed neural networks (PINNs) applied to high-dimensional parametric pa...
By Denis Korolev, Martin Eigel