arXiv Machine Learning July 8, 2026 By Yao Fu, Chunxia Zhang, Junmin Liu, Yihang Jin, Haishan Ye, Yuanao Yang

Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning

Read the original on arXiv Machine Learning →

arXiv:2607. 06151v1 Announce Type: new Abstract: Generalization remains a pivotal challenge in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitting and reduced performance on unseen data.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

benchmarks

arXiv AI

Jul 1

Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization

arXiv:2606. 30813v1 Announce Type: cross Abstract: Deep neural networks with repeated architectural blocks, such as transformers, often exhibit structured relationships across layers that emerge during training.

By Haoming Meng, Anton Sugolov, Vardan Papyan

llms diffusion computer-vision reinforcement-learning

arXiv AI

Jul 22

Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization

arXiv:2607. 18306v1 Announce Type: cross Abstract: Sharpness-Aware Minimization (SAM) improves generalization by minimizing the worst-case loss in a local parameter neighborhood.

By Zhen Huang, Jiaxin Deng, Junbiao Pang

computer-vision fine-tuning

arXiv Machine Learning

1d ago

EMASAM: a Computationally Efficient Sharpness-Aware Minimization via EMA-Guided Perturbations

arXiv:2608. 15105v1 Announce Type: new Abstract: Recent progress in optimization research has highlighted the sharpness of the loss landscape as a key factor in narrowing the generalization gap.

By Tanapat Ratchatorn, Masayuki Tanaka

arXiv AI

Jul 21

Reducing Per-Sample Harm in Stochastic Optimization

arXiv:2607. 16261v1 Announce Type: cross Abstract: Modern optimizers combine gradients from the current mini-batch with historical optimization state, such as momentum or adaptive moments.

By Apostolos Avranas

computer-vision benchmarks

arXiv Machine Learning

Jul 31

Towards Stability of Parameter-Free Optimization

arXiv:2405. 04376v4 Announce Type: replace Abstract: Hyperparameter tuning, particularly the selection of an appropriate learning rate in adaptive gradient training methods, remains a challenge.

By Yijiang Pang, Shuyang Yu, Bao Hoang, Jiayu Zhou

arXiv Machine Learning

Jun 25

Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization

arXiv:2505. 23866v2 Announce Type: replace Abstract: Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving.

By Chengli Tan, Yubo Zhou, Haishan Ye, Guang Dai, Junmin Liu, Zengjie Song, Jiangshe Zhang, Zixiang Zhao, Yunda Hao, Yong Xu

agents robotics safety

Related stories

Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization

Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization

EMASAM: a Computationally Efficient Sharpness-Aware Minimization via EMA-Guided Perturbations

Reducing Per-Sample Harm in Stochastic Optimization

Towards Stability of Parameter-Free Optimization

Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization