Hugging Face Trending Papers
Subspace Levenberg Marquardt Algorithms in Training Neural Networks
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Sep 2
Subspace Levenberg Marquardt Algorithms in Training Neural Networks
arXiv:2609.00789v1 Announce Type: new Abstract: The Levenberg-Marquardt (LM) algorithm is a well-known second-order method for rapid convergence and strong robustness when training small- to medium-s...
arXiv Machine Learning
Aug 27
Adaptive Hybrid Subspace Levenberg Marquardt Algorithm with Adequacy Monitor for Large Scale Least Squares Problems
The paper introduces an Adaptive Hybrid Subspace Levenberg–Marquardt (HSLM) algorithm that tackles large‑scale nonlinear least‑squares problems by building a low‑dimensional subspace from gradient, memory, Krylov‑subspace, and randomized curvature data. It employs a deterministic adequacy monitor to adaptively enrich the subspace and decouples step acceptance from damping adjustment, using Armijo backtracking for step length and a ratio of actual to predicted reduction for damping updates. The authors prove global convergence to stationarity and local linear and superlinear convergence, and demonstrate that HSLM matches the convergence of classical and Krylov‑subspace LM while significantly reducing per‑iteration cost, especially as the parameter dimension increases.
arXiv Machine Learning
Sep 4
Linearized subspace refinement framework to expose hidden accuracy in trained neural networks
The paper introduces Linearized Subspace Refinement (LSR), a post‑training framework that uses the local linearized model of a trained neural network to compute a low‑dimensional correction via a reduced least‑squares problem. LSR is architecture‑agnostic and improves accuracy across tasks such as function approximation, operator learning, physics‑informed fine‑tuning, and noisy inverse problems, often achieving order‑of‑magnitude error reductions. The method reveals that standard training can leave significant accuracy plateaus due to numerical ill‑conditioning, and it offers a subspace rank that balances correction strength, stability, and noise sensitivity.
arXiv Machine Learning
1d ago
Gradient Descent with Stochastic Subspaces via Persistence of Memory
arXiv:2609.18416v1 Announce Type: cross Abstract: Stochastic subspace methods have gained popularity as gradient descent based techniques for large scale optimisation problems, especially in distribu...
arXiv Machine Learning
Aug 7
Alternating Levenberg-Marquardt Training of Physics-Informed Neural Networks with Fourier-Enhanced Features
arXiv:2608. 05892v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) often fail to accurately resolve partial differential equations (PDEs) with high-frequency or multi-scale solutions, as well as strongly nonlinear problems.
Hugging Face Trending Papers
Aug 6
Alternating Levenberg-Marquardt Training of Physics-Informed Neural Networks with Fourier-Enhanced Features
Physics-informed neural networks (PINNs) often fail to accurately resolve partial differential equations (PDEs) with high-frequency or multi-scale solutions, as well as strongly nonlinear problems. Two factors underlie this difficulty: spectral bias, the tendency of neural networks to underfit high-frequency features; and representation-coefficient coupling, the entanglement of representation learning and coefficient fitting within a single nonconvex optimization objective.