arXiv Machine Learning July 16, 2026 By Aristotelis Ballas, Christos Diou

Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning

Read the original on arXiv Machine Learning →

arXiv:2605. 07914v2 Announce Type: replace Abstract: Sharpness-aware and gradient-alignment methods have been shown to improve generalization, however each family of methods targets a single geometric property of the loss landscape, while ignoring the other.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 6

Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics

arXiv:2608. 04382v1 Announce Type: new Abstract: Gradient descent has been of particular interest in modern machine learning beyond sole focus on optimization.

By Han Bao

safety

arXiv AI

Jun 30

Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks

arXiv:2606. 28662v1 Announce Type: cross Abstract: The flatness hypothesis suggests that flatness of the loss landscape, as measured by the eigenvalues of the loss Hessian, correlates with better neural network generalization.

By Yuto Omae, Kazuki Sakai, Yohei Kakimoto, Makoto Sasaki, Yusuke Sakai, Hirotaka Takahashi

arXiv Machine Learning

Jun 25

A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization

arXiv:2602. 02877v2 Announce Type: replace Abstract: This paper studies optimization for a family of problems termed $\textbf{compositional entropic risk minimization}$, in which each data's loss is formulated as a Log-Expectation-Exponential (Log-E-Exp) function.

By Xiyuan Wei, Linli Zhou, Bokun Wang, Chih-Jen Lin, Tianbao Yang

arXiv AI

Jul 1

On the Convergence of Self-Improving Online LLM Alignment

arXiv:2606. 31524v1 Announce Type: cross Abstract: The Self-Improving Alignment (SAIL) algorithm addresses distribution shift by reducing a bilevel formulation of the problem to an efficient, single-level method.

By Xudong Wu, Pangpang Liu, Vaneet Aggarwal, Jiayu Chen

llms benchmarks safety

arXiv Machine Learning

Jun 19

Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima

arXiv:2606. 20469v1 Announce Type: new Abstract: A widely held intuition in deep learning is that stochastic gradient descent (SGD) implicitly favors flat minima and that flat minima generalize better, but standard Euclidean measures of flatness such as the trace or maximum eigenvalue of the loss Hessian are not invariant under reparametrizations that preserve the network function, which undermines the theoretical foundations of this narrative.

By Md Sakir Ahmed, Kumaresh Sarmah, Hemen Dutta

safety

arXiv Machine Learning

Jul 16

AvAtar: Learning to Align via Active Optimal Transport

arXiv:2605. 24395v2 Announce Type: replace Abstract: Alignment plays a fundamental role in many machine learning problems, such as multi-network analysis, multimodal learning, and point cloud registration.

By Qi Yu, Ruizhong Qiu, Zhichen Zeng, My T. Thai, Huan Liu, Hanghang Tong

multimodal safety

Related stories

Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics

Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks

A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization

On the Convergence of Self-Improving Online LLM Alignment

Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima

AvAtar: Learning to Align via Active Optimal Transport