arXiv Machine Learning June 3, 2026 By Anherutowa Calvo

Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent

Read the original on arXiv Machine Learning →

arXiv:2606. 02596v1 Announce Type: new Abstract: The curvature exponent $\alpha$ in $h_k \propto \sigma_k^\alpha$ -- governing how Hessian eigenvalues scale with gradient singular values -- varies systematically across layer types ($\alpha \approx 2$ for convolutions, $\approx 1$ for transformer attention, $< 1$ for MLP up-projections).

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

5d ago

Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

arXiv:2608. 13335v1 Announce Type: new Abstract: Neural networks trained by gradient descent on a smooth cost function can nevertheless learn in steps: the cost holds on long plateaus and then drops abruptly.

By Liu Ziyin, Yizhou Xu, Tomaso Poggio, Isaac Chuang

arXiv Machine Learning

Aug 6

The Loss Does Not See the Basis, but Adam Does

arXiv:2608. 05136v1 Announce Type: new Abstract: Gradient descent on a factored model $W = UV^\top$ is implicitly biased toward low-rank solutions, while Adam, starting from the same small initialization, is not.

By Devender Singh

llms safety

arXiv Machine Learning

Jul 10

Coupling-Robust Accuracy in Multiphysics Physics Informed Neural Networks via Kronecker-Preconditioned Optimization

arXiv:2605. 23391v2 Announce Type: replace Abstract: Physics-informed neural networks (PINNs) for coupled multiphysics systems suffer systematic accuracy degradation as inter-equation coupling strengthens.

By Youngjae Park, Jaemin Kim, Junghwa Hong

benchmarks

arXiv Machine Learning

Jul 7

Directional Curvature from Armijo Backtracking: A Low-Cost Sharpness Probe and a Calibration-Free Learning-Rate Safeguard for Adam

arXiv:2607. 03998v1 Announce Type: new Abstract: The local sharpness of the loss, the top Hessian eigenvalue $\lambda_1$, determines the largest stable gradient step, but measuring it normally requires Lanczos or Hessian-vector iterations.

By Ashmitha R, J\"org Frochte

benchmarks safety

arXiv Machine Learning

Jun 9

On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature

arXiv:2602. 05600v2 Announce Type: replace Abstract: Stochastic Gradient Descent (SGD) introduces anisotropic noise that is correlated with the local curvature of the loss landscape, thereby biasing optimization toward flat minima.

By Yikuan Zhang, Ning Yang, Yuhai Tu

arXiv Machine Learning

Jul 7

The Multiscale Single-Index Model: A Stylized Model for Hierarchical Feature Learning

arXiv:2607. 03347v1 Announce Type: new Abstract: We consider the Multiscale Single-Index Model (MSIM), first introduced in \cite{oymak2021learning}, as a stylized model for hierarchical learning with \emph{scale separation}.

By Joan Bruna

Related stories

Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

The Loss Does Not See the Basis, but Adam Does

Coupling-Robust Accuracy in Multiphysics Physics Informed Neural Networks via Kronecker-Preconditioned Optimization

Directional Curvature from Armijo Backtracking: A Low-Cost Sharpness Probe and a Calibration-Free Learning-Rate Safeguard for Adam

On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature

The Multiscale Single-Index Model: A Stylized Model for Hierarchical Feature Learning