Hugging Face Trending Papers
How Edge of Stability Hinders SCAFFOLD in Federated Optimization
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Aug 27
How Edge of Stability Hinders SCAFFOLD in Federated Optimization
The paper investigates why the SCAFFOLD algorithm, designed to be robust to data heterogeneity in federated learning, often underperforms compared to the simpler FedAvg. It identifies the presence of Edge of Stability (EoS) dynamics and progressive sharpening as key factors, showing that both algorithms exhibit EoS behavior across various architectures and hyperparameters. Crucially, the study finds that at the EoS, SCAFFOLD’s ability to estimate the global gradient deteriorates, as indicated by a weakened correlation between sharpness and gradient estimation error, explaining its limited practical advantage.
arXiv AI
Oct 12
Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning
arXiv:2505. 11304v4 Announce Type: replace-cross Abstract: Federated learning (FL) commonly involves clients with diverse communication and computational capabilities.
arXiv AI
Jun 2
Stability Analysis of Sharpness-Aware Minimization
arXiv:2301. 06308v2 Announce Type: replace-cross Abstract: Sharpness-aware minimization (SAM) is a training method that seeks to find flat minima in deep learning, resulting in state-of-the-art performance across various domains.
arXiv Machine Learning
Jun 4
Edge of Stability Selectively Shapes Learning Across the Data Distribution
arXiv:2606. 04212v1 Announce Type: new Abstract: Existing analyses of the edge of stability (EoS) treat it as a global property of optimization.
arXiv Machine Learning
Sep 14
High-Probability Convergence of SGD via Batched Updates
The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.
arXiv AI
Jul 21
Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization
arXiv:2605. 29547v2 Announce Type: replace-cross Abstract: Deep learning optimization relies heavily on the assumption of smooth loss landscapes, a condition systematically violated by modern architectures due to non-smooth components such as ReLU activations and quantization operators.