arXiv AI
How Much Regularization Survives Averaging? Update Masking in Federated Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 10
Range Penalization: Theoretical Insights with Applications in Federated Learning
arXiv:2606. 10916v1 Announce Type: cross Abstract: This paper introduces range regularization for federated learning with linear systematic components to enhance statistical accuracy and induce cross-client regularity conducive to quantization, coding, and resource efficiency.
arXiv Machine Learning
Jul 7
FedFFT: Taming Client Drift in Federated SAM via Spectral Perturbation Filtering
arXiv:2607. 04170v1 Announce Type: new Abstract: Federated Learning (FL) enables decentralized training without data sharing, but suffers from statistical heterogeneity across clients, leading to client drift, poor generalization, and sharp minima compared to centralized training.
arXiv Machine Learning
Jun 24
FAIRVAR: Fair Federated Learning via Variance Regularization
arXiv:2508. 12042v3 Announce Type: replace Abstract: Federated learning (FL) allows collaborative training of machine learning models across multiple parties without sharing raw data.
arXiv Machine Learning
Jul 3
Class-Grouped Normalized Momentum and Faster Hyperparameter Exploration to Tackle Class Imbalance in Federated Learning
arXiv:2607. 01474v1 Announce Type: new Abstract: Class imbalance poses a critical challenge in federated learning (FL), where underrepresented classes suffer from poor predictive performance yet cannot be addressed by standard centralized techniques due to privacy and heterogeneity constraints.
arXiv Machine Learning
Jul 17
What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity
arXiv:2607. 14731v1 Announce Type: new Abstract: Local SGD, also known as Federated Averaging, is a widely used distributed optimization algorithm.
arXiv Machine Learning
Aug 27
How Edge of Stability Hinders SCAFFOLD in Federated Optimization
The paper investigates why the SCAFFOLD algorithm, designed to be robust to data heterogeneity in federated learning, often underperforms compared to the simpler FedAvg. It identifies the presence of Edge of Stability (EoS) dynamics and progressive sharpening as key factors, showing that both algorithms exhibit EoS behavior across various architectures and hyperparameters. Crucially, the study finds that at the EoS, SCAFFOLD’s ability to estimate the global gradient deteriorates, as indicated by a weakened correlation between sharpness and gradient estimation error, explaining its limited practical advantage.