arXiv Machine Learning June 16, 2026 By Tolga Dimlioglu, Anna Choromanska

Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning

Read the original on arXiv Machine Learning →

arXiv:2507. 20424v3 Announce Type: replace Abstract: We study centralized distributed data parallel training of deep neural networks (DNNs), aiming to improve the trade-off between communication efficiency and model performance of the local gradient methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 8

Flatland: The Adventures of Gradient Descent with Large Step Sizes

arXiv:2606. 06722v1 Announce Type: new Abstract: The training of neural networks often entails objective functions that are not globally $L$-smooth.

By Leonardo Galli, Curtis Fox, Wiebke Bartolomaeus, Mark Schmidt, Holger Rauhut

arXiv Machine Learning

Jun 16

Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning

arXiv:2512. 12737v2 Announce Type: replace Abstract: Decentralized federated learning (DFL) enables collaborative model training without a central server, but converges slowly under statistical heterogeneity.

By Li Xia

data-engineering

arXiv Machine Learning

Jun 4

When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

arXiv:2606. 04476v1 Announce Type: new Abstract: In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.

By Berk Tinaz, Changzhi Xie, Mahdi Soltanolkotabi

safety

arXiv Machine Learning

Aug 11

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

arXiv:2608. 09523v1 Announce Type: new Abstract: Deep neural network (DNN) training with stochastic gradient descent (SGD) and its variants achieves strong empirical performance, yet classical optimization theory does not fully explain this success.

By Binchuan Qi

safety

arXiv Machine Learning

1d ago

On the Escaping Efficiency of Distributed Adversarial Training Algorithms

The paper compares distributed adversarial training algorithms—both centralized and decentralized—within multi‑agent learning environments. It introduces a theoretical framework to analyze how efficiently these algorithms escape local minima, a property linked to model flatness and robustness. The study finds that with small perturbation bounds and large batch sizes, decentralized methods (consensus and diffusion) escape local minima faster than centralized ones, but this advantage may diminish as attack strength increases.

By Ying Cao, Kun Yuan, Ali H. Sayed

agents diffusion safety

arXiv Machine Learning

Aug 11

Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks

arXiv:2502. 11152v4 Announce Type: replace-cross Abstract: The optimization foundations of deep linear networks have recently received significant attention.

By Po Chen, Rujun Jiang, Peng Wang

One email a morning, machine-written

Related stories

Flatland: The Adventures of Gradient Descent with Large Step Sizes

Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning

When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

On the Escaping Efficiency of Distributed Adversarial Training Algorithms

Error Bound Analysis for the Regularized Loss of Deep Linear Neural Networks