arXiv Machine Learning July 8, 2026 By Yao Fu, Chunxia Zhang, Junmin Liu, Yihang Jin, Haishan Ye, Yuanao Yang

Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning

Read the original on arXiv Machine Learning →

arXiv:2607. 06151v1 Announce Type: new Abstract: Generalization remains a pivotal challenge in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitting and reduced performance on unseen data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

benchmarks

arXiv Machine Learning

Aug 26

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods

arXiv:2402.11215v4 Announce Type: replace Abstract: The choice of batch size in minibatch stochastic gradient optimization is critical for both optimization and generalization performance in large-sc...

By Tim Tsz-Kit Lau, Han Liu, Mladen Kolar

arXiv AI

Jul 1

Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization

arXiv:2606. 30813v1 Announce Type: cross Abstract: Deep neural networks with repeated architectural blocks, such as transformers, often exhibit structured relationships across layers that emerge during training.

By Haoming Meng, Anton Sugolov, Vardan Papyan

llms diffusion computer-vision reinforcement-learning

arXiv AI

Jul 22

Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization

arXiv:2607. 18306v1 Announce Type: cross Abstract: Sharpness-Aware Minimization (SAM) improves generalization by minimizing the worst-case loss in a local parameter neighborhood.

By Zhen Huang, Jiaxin Deng, Junbiao Pang

computer-vision fine-tuning

arXiv Machine Learning

Aug 18

EMASAM: a Computationally Efficient Sharpness-Aware Minimization via EMA-Guided Perturbations

arXiv:2608. 15105v1 Announce Type: new Abstract: Recent progress in optimization research has highlighted the sharpness of the loss landscape as a key factor in narrowing the generalization gap.

By Tanapat Ratchatorn, Masayuki Tanaka

arXiv AI

Jul 21

Reducing Per-Sample Harm in Stochastic Optimization

arXiv:2607. 16261v1 Announce Type: cross Abstract: Modern optimizers combine gradients from the current mini-batch with historical optimization state, such as momentum or adaptive moments.

By Apostolos Avranas

computer-vision benchmarks

arXiv Machine Learning

Jul 31

Towards Stability of Parameter-Free Optimization

arXiv:2405. 04376v4 Announce Type: replace Abstract: Hyperparameter tuning, particularly the selection of an appropriate learning rate in adaptive gradient training methods, remains a challenge.

By Yijiang Pang, Shuyang Yu, Bao Hoang, Jiayu Zhou

One email a morning, machine-written

Related stories

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods

Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization

Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization

EMASAM: a Computationally Efficient Sharpness-Aware Minimization via EMA-Guided Perturbations

Reducing Per-Sample Harm in Stochastic Optimization

Towards Stability of Parameter-Free Optimization