arXiv Machine Learning
One Size Does Not Fit All: Curvature-Aware Adaptive Perturbation for SAM
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 2
Stability Analysis of Sharpness-Aware Minimization
arXiv:2301. 06308v2 Announce Type: replace-cross Abstract: Sharpness-aware minimization (SAM) is a training method that seeks to find flat minima in deep learning, resulting in state-of-the-art performance across various domains.
arXiv AI
Aug 26
Across the Loss Landscape with Progressive Growth
arXiv:2608.24568v1 Announce Type: cross Abstract: Deep neural networks generalize well despite their highly nonconvex, overparameterized loss landscapes, a phenomenon often associated with the geomet...
arXiv Machine Learning
Jul 8
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
arXiv:2607. 06151v1 Announce Type: new Abstract: Generalization remains a pivotal challenge in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitting and reduced performance on unseen data.
arXiv AI
Jul 22
Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization
arXiv:2607. 18306v1 Announce Type: cross Abstract: Sharpness-Aware Minimization (SAM) improves generalization by minimizing the worst-case loss in a local parameter neighborhood.
arXiv Machine Learning
Jun 2
Exploiting weight-space symmetries for approximating curvature
arXiv:2606. 00442v1 Announce Type: new Abstract: Many machine learning techniques rely on approximating a loss function's curvature, but this is notoriously hard to do at the scale of modern deep networks.
arXiv Computer Vision
Sep 15
Sparsity-Adaptive Sharpness-Aware Minimization
The paper introduces Sparsity-Adaptive Sharpness-Aware Minimization (SA‑SAM), a method that adjusts the perturbation radius in sharpness-aware training to remain consistent as model sparsity increases. It also evaluates a Magnitude‑Weighted Hessian (MWH) importance metric derived from second‑order analysis. Experiments on CIFAR‑10‑C, CIFAR‑100‑C, and ImageNet‑100‑C show that SA‑SAM improves corruption robustness at 80–90% sparsity while maintaining clean accuracy, and the study reports inference throughput at deployment‑relevant sparsity levels.