arXiv Machine Learning August 10, 2026 By Grigory Malinovsky

Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization

Read the original on arXiv Machine Learning →

arXiv:2608. 06563v1 Announce Type: new Abstract: Machine learning and optimization have advanced together, with practical demands motivating new theory and theoretical breakthroughs enabling new applications.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

fine-tuning

arXiv Machine Learning

Jul 15

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication

arXiv:2405. 11667v2 Announce Type: replace Abstract: Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD.

By Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari, Lingxiao Wang, Sebastian U. Stich, Ziheng Cheng, Nirmit Joshi, Nathan Srebro

arXiv Machine Learning

Jun 10

FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection

arXiv:2604. 24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments.

By Yutong He, Zhengyang Huang, Jiahe Geng, Kun Yuan

arXiv Machine Learning

Jun 9

TAMUNA: Doubly Accelerated Distributed Optimization under Partial Participation

arXiv:2302. 09832v4 Announce Type: replace Abstract: In distributed optimization and federated learning, slow and costly communication between parallel devices and the central server constitutes the primary bottleneck.

By Laurent Condat, Ivan Agarsk\'y, Grigory Malinovsky, Peter Richt\'arik

arXiv Machine Learning

Jun 2

Local MixVR: Breaking the Communication-Sample Dependence in Distributed Learning

arXiv:2606. 01128v1 Announce Type: new Abstract: Communication overhead is a crucial bottleneck in scalable distributed learning.

By Tehila Dahan, Bassel Hamoud, Roie Reshef, Martin Jaggi, Kfir Y. Levy

benchmarks

arXiv Machine Learning

Jul 14

First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

arXiv:2603. 05774v2 Announce Type: replace Abstract: This paper addresses the distributed stochastic minimax optimization problem subject to stochastic constraints.

By Zhankun Luo, Antesh Upadhyay, Sang Bin Moon, Abolfazl Hashemi

reinforcement-learning

arXiv Machine Learning

Jun 2

Dimensionality Reduction for Robust Federated Learning: A Theoretical Analysis and Convergence Guarantee

arXiv:2605. 28335v2 Announce Type: replace Abstract: Federated Learning (FL) enables multiple clients to collaboratively train models without sharing raw data, but it is highly vulnerable to Byzantine attacks.

By Shiyuan Zuo, Jiashuo Li, Rongfei Fan, Han Hu, Jie Xu

rag benchmarks

Related stories

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication

FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection

TAMUNA: Doubly Accelerated Distributed Optimization under Partial Participation

Local MixVR: Breaking the Communication-Sample Dependence in Distributed Learning

First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

Dimensionality Reduction for Robust Federated Learning: A Theoretical Analysis and Convergence Guarantee