arXiv Machine Learning
RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 14
High-Probability Convergence of SGD via Batched Updates
The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.
arXiv Machine Learning
1d ago
On the Escaping Efficiency of Distributed Adversarial Training Algorithms
The paper compares distributed adversarial training algorithms—both centralized and decentralized—within multi‑agent learning environments. It introduces a theoretical framework to analyze how efficiently these algorithms escape local minima, a property linked to model flatness and robustness. The study finds that with small perturbation bounds and large batch sizes, decentralized methods (consensus and diffusion) escape local minima faster than centralized ones, but this advantage may diminish as attack strength increases.
arXiv Machine Learning
Sep 10
Approaching the Harm of Gradient Attacks While Only Flipping Labels
The paper investigates the impact of label‑flipping attacks on distributed machine learning, where an adversary can only flip a limited number of training labels. It formalizes the attack as a per‑round constrained optimization problem, derives a greedy label‑selection rule for logistic regression, and shows that this rule is provably optimal under mean aggregation. Experiments demonstrate that optimized label flipping can significantly degrade model accuracy, outperforming random flips, and that the attack transfers to other robust aggregators such as coordinate‑wise median and trimmed mean.
arXiv Machine Learning
Jun 2
Dimensionality Reduction for Robust Federated Learning: A Theoretical Analysis and Convergence Guarantee
arXiv:2605. 28335v2 Announce Type: replace Abstract: Federated Learning (FL) enables multiple clients to collaboratively train models without sharing raw data, but it is highly vulnerable to Byzantine attacks.
arXiv Machine Learning
Jun 8
Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization
arXiv:2606. 07496v1 Announce Type: new Abstract: Decentralized stochastic optimization is a fundamental paradigm for large-scale learning over networks, where agents communicate only with their neighbors and no central coordinator is required.
arXiv Machine Learning
Aug 10
Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization
arXiv:2608. 06563v1 Announce Type: new Abstract: Machine learning and optimization have advanced together, with practical demands motivating new theory and theoretical breakthroughs enabling new applications.