arXiv Machine Learning
Revisiting Distributed Sign-Based Variance Reduction
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 12
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization
arXiv:2310. 15976v4 Announce Type: replace Abstract: signSGD is attractive in nonconvex optimization because it communicates sign-valued rather than full-precision gradients.
arXiv Machine Learning
Jun 4
Near-Optimal Decentralized Stochastic Convex Optimization over Networks
arXiv:2606. 04757v1 Announce Type: cross Abstract: We study decentralized stochastic smooth convex optimization, where $M$ workers minimize an average objective using local stochastic gradients and neighbor-only communication over a fixed gossip network.
arXiv Machine Learning
Jun 8
Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization
arXiv:2606. 07496v1 Announce Type: new Abstract: Decentralized stochastic optimization is a fundamental paradigm for large-scale learning over networks, where agents communicate only with their neighbors and no central coordinator is required.
arXiv Machine Learning
Jun 3
Decentralized Stochastic Nonconvex Optimization under the $(L_0,L_1)$-Smoothness
arXiv:2509. 08726v3 Announce Type: replace-cross Abstract: This paper focuses on the decentralized stochastic optimization problem $f(\mathbf{x})=\frac{1}{m}\sum_{i=1}^m f_i(\mathbf{x})$ over a connected network of $n$ agents, where each local function has the form of $f_i(\mathbf{x}) = {\mathbb E}\left[F(\mathbf{x};{\boldsymbol \xi}_i)\right]$ which satisfies the $(L_0,L_1)$-smooth condition but possibly nonconvex and each random variable ${\boldsymbol \xi}_i$ follows distribution ${\mathcal D}_i$.
arXiv Computer Vision
Aug 26
Adaptive Extrapolated Proximal Gradient Methods with Variance Reduction for Composite Nonconvex Finite-Sum Minimization
arXiv:2502.21099v3 Announce Type: replace-cross Abstract: This paper proposes {\sf AEPG-SPIDER}, an Adaptive Extrapolated Proximal Gradient (AEPG) method with variance reduction for minimizing compos...
arXiv Machine Learning
3d ago
Solving Finite-sum Coupled Compositional Optimization via Multi-block-Single-probe Estimator
arXiv:2609.15723v1 Announce Type: new Abstract: Traditional variance reduction methods (e.g., SPIDER, SARAH, STORM) have been extensively investigated for improving the convergence rates of stochasti...