arXiv AI
Fast and Efficient Asynchronous Gossip Algorithm for Robust and Non-Smooth Convex Decentralized Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 8
Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization
arXiv:2606. 07496v1 Announce Type: new Abstract: Decentralized stochastic optimization is a fundamental paradigm for large-scale learning over networks, where agents communicate only with their neighbors and no central coordinator is required.
arXiv Machine Learning
Aug 11
Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum
arXiv:2504. 12742v2 Announce Type: replace Abstract: Decentralized Federated Learning (DFL) enables collaborative model training without relying on a central server.
arXiv Machine Learning
Jun 4
Near-Optimal Decentralized Stochastic Convex Optimization over Networks
arXiv:2606. 04757v1 Announce Type: cross Abstract: We study decentralized stochastic smooth convex optimization, where $M$ workers minimize an average objective using local stochastic gradients and neighbor-only communication over a fixed gossip network.
arXiv Machine Learning
Jun 10
FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection
arXiv:2604. 24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments.
arXiv Machine Learning
Sep 7
Resilience Beyond Stationary Client Unavailability: Unlocking Efficient and Unbiased Federated Learning
The paper introduces FedSWE, a federated learning algorithm designed to handle non‑stationary and heterogeneous client availability without requiring prior real‑time knowledge of which devices are online. FedSWE compensates for missed computations, stabilizes global updates, and mixes local updates through implicit gossiping, all while adding only modest memory and computational overhead. The authors prove that FedSWE converges to a stationary point for non‑convex objectives and achieves linear speedup in certain scenarios, and they validate these claims with experiments on real‑world datasets featuring diverse client unavailability patterns.
arXiv Machine Learning
Jul 15
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
arXiv:2405. 11667v2 Announce Type: replace Abstract: Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD.