arXiv Machine Learning July 10, 2026 By Marcel K\"uhn, Bernd Rosenow

Explaining Near-Zero Hessian Eigenvalues Through Approximate Symmetries in Neural Networks

Read the original on arXiv Machine Learning →

arXiv:2607. 07845v1 Announce Type: new Abstract: The Hessian of the training loss governs the local geometry of the loss landscape, yet despite existing explanations for its largest eigenvalues, the origin of the vast multitude of vanishingly small eigenvalues remains elusive.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

5d ago

Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

arXiv:2608. 13335v1 Announce Type: new Abstract: Neural networks trained by gradient descent on a smooth cost function can nevertheless learn in steps: the cost holds on long plateaus and then drops abruptly.

By Liu Ziyin, Yizhou Xu, Tomaso Poggio, Isaac Chuang

arXiv Machine Learning

Jul 7

A Complete Symmetry Classification of Shallow ReLU Networks

arXiv:2604. 14037v2 Announce Type: replace Abstract: Parameter space is not function space for neural network architectures.

By Pranavkrishnan Ramakrishnan

arXiv Machine Learning

Jun 24

A Theory of Saddle Escape in Deep Nonlinear Networks

arXiv:2605. 01288v3 Announce Type: replace Abstract: In deep networks with small initialization, training exhibits long plateaus separated by sharp feature-acquisition transitions.

By Divit Rawal, Michael R. DeWeese

arXiv Machine Learning

Jun 4

Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability

arXiv:2606. 04754v1 Announce Type: new Abstract: Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformations that leave the realized function unchanged.

By Vincent B\"urgin, Daniel Herbst, Ya-Wei Eileen Lin, Stefanie Jegelka

safety

arXiv Machine Learning

Jul 21

Effects of width-dependent model hyperparameters and $\ell_2$-regularization on the loss landscape of two-layer ReLU networks

arXiv:2607. 16720v1 Announce Type: new Abstract: Understanding deep neural networks remains a central challenge in machine learning.

By Haruka Eshima, Makoto Yamada

arXiv Machine Learning

Jul 7

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

arXiv:2607. 03613v1 Announce Type: new Abstract: We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression.

By Shuang Liang, Tom Jacobs, Guido Mont\'ufar

safety

Related stories

Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws

A Complete Symmetry Classification of Shallow ReLU Networks

A Theory of Saddle Escape in Deep Nonlinear Networks

Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability

Effects of width-dependent model hyperparameters and $\ell_2$-regularization on the loss landscape of two-layer ReLU networks

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse