arXiv Machine Learning June 19, 2026 By Md Sakir Ahmed, Kumaresh Sarmah, Hemen Dutta

Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima

Read the original on arXiv Machine Learning →

arXiv:2606. 20469v1 Announce Type: new Abstract: A widely held intuition in deep learning is that stochastic gradient descent (SGD) implicitly favors flat minima and that flat minima generalize better, but standard Euclidean measures of flatness such as the trace or maximum eigenvalue of the loss Hessian are not invariant under reparametrizations that preserve the network function, which undermines the theoretical foundations of this narrative.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

safety

arXiv Machine Learning

Jun 9

On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature

arXiv:2602. 05600v2 Announce Type: replace Abstract: Stochastic Gradient Descent (SGD) introduces anisotropic noise that is correlated with the local curvature of the loss landscape, thereby biasing optimization toward flat minima.

By Yikuan Zhang, Ning Yang, Yuhai Tu

arXiv Machine Learning

Jun 18

Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds

arXiv:2606. 18306v1 Announce Type: new Abstract: Gaussian width is a central geometric complexity measure in high-dimensional probability, compressed sensing, convex optimization, and learning theory.

By Vu Khac Ky

arXiv Machine Learning

Jul 10

Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima

arXiv:2607. 08380v1 Announce Type: new Abstract: An important quantity in the theory of gradient descent (GD) is the \emph{sharpness}, defined as the largest eigenvalue of the objective Hessian.

By Lachlan Ewen MacDonald, Ren\'e Vidal

arXiv Machine Learning

Jun 8

Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization

arXiv:2604. 03146v2 Announce Type: replace-cross Abstract: We study high-dimensional convex empirical risk minimization (ERM) under general non-Gaussian data designs.

By Chiheb Yaakoubi, Cosme Louart, Malik Tiomoko, Zhenyu Liao

arXiv Machine Learning

Jul 30

Minimax-Optimal Generalization Bounds for Smooth Deep Neural Networks Trained by (Stochastic) Gradient Descent

arXiv:2606. 06772v2 Announce Type: replace-cross Abstract: Characterizing the optimization dynamics and statistical performance of over-parameterized deep neural networks (DNNs) remains a central challenge in understanding the remarkable success of deep learning.

By Junyu Zhou, Puyu Wang, Dennis Wagner, Yunwen Lei, Marius Kloft, Yiming Ying

arXiv Machine Learning

Jul 24

Fisher Widths: Local Learning Geometry and Anisotropic Recovery

arXiv:2607. 20578v1 Announce Type: new Abstract: We study Gaussian-width complexity on statistical manifolds through a pair of functionals: the primal Fisher width $w_G(T) = w(G^{1/2}T)$, induced by the Fisher metric, and the inverse-Fisher width $w_{G^{-1}}(T) = w(G^{-1/2}T)$, induced by the inverse Fisher metric.

By Vu Khac Ky

efficiency

Related stories

On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature

Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds

Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima

Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization

Minimax-Optimal Generalization Bounds for Smooth Deep Neural Networks Trained by (Stochastic) Gradient Descent

Fisher Widths: Local Learning Geometry and Anisotropic Recovery