arXiv Machine Learning July 7, 2026 By Guillaume Braun, Han Bao, Wei Huang, Masaaki Imaizumi

Spectral Gradient Descent Mitigates Anisotropy-Driven Misalignment: A Case Study in Phase Retrieval

Read the original on arXiv Machine Learning →

arXiv:2601. 22652v2 Announce Type: replace-cross Abstract: Spectral gradient methods, such as the Muon optimizer, modify gradient updates by preserving directional information while discarding scale, and have shown strong empirical performance in deep learning.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

safety

arXiv Machine Learning

Jun 30

Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localization

arXiv:2606. 30226v1 Announce Type: new Abstract: Hessian spectral properties are a standard tool in analysing neural-network training, with eigenvalues linked to sharpness, generalization, and optimization dynamics.

By Marcelina Marjankowska, Valerio Modugno, Paolo Barucca

arXiv Machine Learning

Jul 7

SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity

arXiv:2607. 04189v1 Announce Type: new Abstract: Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global convergence.

By Liyang Yuan, Yibo Yang, Dandan Guo, Peter Richtarik, Zhouchen Lin

benchmarks safety

arXiv Machine Learning

Jun 4

Muon in Associative Memory Learning: Training Dynamics and Scaling Laws

arXiv:2602. 05725v3 Announce Type: replace Abstract: Muon updates matrix parameters via the matrix sign of the gradient and has shown strong empirical gains, yet its dynamics and scaling behavior remain unclear in theory.

By Binghui Li, Kaifei Wang, Han Zhong, Pinyan Lu, Liwei Wang

llms safety

arXiv Machine Learning

Jul 28

Mini-batch Noise Lowers Sharpness via Dominant-Subspace Fluctuations

arXiv:2607. 23012v1 Announce Type: new Abstract: During SGD training, the gradients often align strongly with the dominant subspace spanned by the top-$k$ eigenvectors of the Hessian of the loss.

By Junho So, Dongwook Shin

arXiv Machine Learning

Jul 21

Orthogonal Gradient Constraints Shape Noisy-Label Memorization Dynamics

arXiv:2607. 16231v1 Announce Type: new Abstract: Modern neural networks can fit corrupted training labels, making noisy-label learning a useful setting for studying memorization-driven overfitting.

By Richard Mai

computer-vision

arXiv Machine Learning

Jun 11

Simplicity Suffices for Parameter Noise Injection in Stochastic Gradient Descent

arXiv:2606. 12054v1 Announce Type: new Abstract: Injecting noise into the optimization process is a well-established technique for improving the training and generalization of deep neural networks.

By Benjamin Leblanc, Louis-Jacob Lebel, Teddy Kana, Richard Kamel

Related stories

Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localization

SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity

Muon in Associative Memory Learning: Training Dynamics and Scaling Laws

Mini-batch Noise Lowers Sharpness via Dominant-Subspace Fluctuations

Orthogonal Gradient Constraints Shape Noisy-Label Memorization Dynamics

Simplicity Suffices for Parameter Noise Injection in Stochastic Gradient Descent