arXiv Machine Learning June 16, 2026 By Tolga Dimlioglu, Anna Choromanska

Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning

Read the original on arXiv Machine Learning →

arXiv:2507. 20424v3 Announce Type: replace Abstract: We study centralized distributed data parallel training of deep neural networks (DNNs), aiming to improve the trade-off between communication efficiency and model performance of the local gradient methods.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 8

Flatland: The Adventures of Gradient Descent with Large Step Sizes

arXiv:2606. 06722v1 Announce Type: new Abstract: The training of neural networks often entails objective functions that are not globally $L$-smooth.

By Leonardo Galli, Curtis Fox, Wiebke Bartolomaeus, Mark Schmidt, Holger Rauhut

arXiv Machine Learning

Jun 16

Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning

arXiv:2512. 12737v2 Announce Type: replace Abstract: Decentralized federated learning (DFL) enables collaborative model training without a central server, but converges slowly under statistical heterogeneity.

By Li Xia

data-engineering

arXiv Machine Learning

Jun 4

When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

arXiv:2606. 04476v1 Announce Type: new Abstract: In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.

By Berk Tinaz, Changzhi Xie, Mahdi Soltanolkotabi

safety

arXiv Machine Learning

Aug 11

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

arXiv:2608. 09523v1 Announce Type: new Abstract: Deep neural network (DNN) training with stochastic gradient descent (SGD) and its variants achieves strong empirical performance, yet classical optimization theory does not fully explain this success.

By Binchuan Qi

safety

arXiv Machine Learning

Aug 11

Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks

arXiv:2502. 11152v4 Announce Type: replace-cross Abstract: The optimization foundations of deep linear networks have recently received significant attention.

By Po Chen, Rujun Jiang, Peng Wang

arXiv AI

Jun 2

Stability Analysis of Sharpness-Aware Minimization

arXiv:2301. 06308v2 Announce Type: replace-cross Abstract: Sharpness-aware minimization (SAM) is a training method that seeks to find flat minima in deep learning, resulting in state-of-the-art performance across various domains.

By Hoki Kim, Jinseong Park, Yujin Choi, Jaewook Lee

diffusion benchmarks

Related stories

Flatland: The Adventures of Gradient Descent with Large Step Sizes

Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning

When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks

Stability Analysis of Sharpness-Aware Minimization