arXiv Machine Learning
Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
5d ago
Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression
Agnostic federated learning (AFL) seeks a model that performs reliably across $m$ heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency by reducing the num...
arXiv Machine Learning
Jun 8
Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization
arXiv:2606. 07496v1 Announce Type: new Abstract: Decentralized stochastic optimization is a fundamental paradigm for large-scale learning over networks, where agents communicate only with their neighbors and no central coordinator is required.
arXiv Machine Learning
Sep 17
Revisiting Distributed Sign-Based Variance Reduction
The paper addresses bias introduced by aggregating local signs in distributed sign-based variance reduction methods, which hampers optimal convergence rates. By proposing an unbiased compression of recursive gradient increments to track the global gradient at the server, the authors achieve optimal convergence rates for both nonconvex stochastic and finite-sum optimization. They provide specific rate bounds for α-norms and demonstrate matching sample complexities to centralized settings for finite-sum problems.
arXiv AI
Jul 22
Federated Lightweight Fine-Tuning
arXiv:2607. 18343v1 Announce Type: cross Abstract: Federated fine-tuning is bottlenecked by communication: FedAvg and pseudo-gradient schemes transmit a payload that scales with the model, and gradient compression shrinks it by only a constant factor.
arXiv AI
Jul 23
Distributed Optimization via Energy Conservation Laws in Dilated Coordinates
arXiv:2409. 19279v2 Announce Type: replace-cross Abstract: Continuous-time models can reveal accelerated structures in distributed optimization, but their rates need not survive direct discretization.
arXiv Machine Learning
Jun 9
TAMUNA: Doubly Accelerated Distributed Optimization under Partial Participation
arXiv:2302. 09832v4 Announce Type: replace Abstract: In distributed optimization and federated learning, slow and costly communication between parallel devices and the central server constitutes the primary bottleneck.