arXiv AI
A Function-Centric Perspective on Flat and Sharp Minima
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 9
Do Flatter Minima Drive Better Generalization? An Algorithmic Separation in Grokking
The paper investigates whether flatter loss landscapes drive better generalization in the context of Grokking. It shows that sharpness‑aware minimization (SAM) alone does not reliably induce the transition to generalization, but when combined with weight decay, SAM can accelerate this transition up to four times faster. The authors provide a theoretical analysis using a minimal two‑layer ReLU model, demonstrating that flatness alone cannot distinguish memorizing from generalizing solutions, while weight decay favors generalization and a local stability window explains SAM’s effect.
arXiv AI
Jul 28
Are Flat Minima an Illusion?
arXiv:2605. 05209v2 Announce Type: replace-cross Abstract: Flat minima are an account of why deep networks generalise.
arXiv Machine Learning
Jul 8
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
arXiv:2607. 06151v1 Announce Type: new Abstract: Generalization remains a pivotal challenge in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitting and reduced performance on unseen data.
arXiv Machine Learning
Aug 24
Generalization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark
arXiv:2602.01718v2 Announce Type: replace Abstract: Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systematic benc...
arXiv Machine Learning
Jun 19
Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima
arXiv:2606. 20469v1 Announce Type: new Abstract: A widely held intuition in deep learning is that stochastic gradient descent (SGD) implicitly favors flat minima and that flat minima generalize better, but standard Euclidean measures of flatness such as the trace or maximum eigenvalue of the loss Hessian are not invariant under reparametrizations that preserve the network function, which undermines the theoretical foundations of this narrative.
arXiv Machine Learning
Jun 30
How Far Can Sharpness and Complexity Jointly Explain Generalization?
arXiv:2606. 29043v1 Announce Type: new Abstract: Sharpness and complexity are two central factors in the generalization analysis of deep neural networks.