arXiv Machine Learning
Gradient Descent with Stochastic Subspaces via Persistence of Memory
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 30
Geometrically Principled Randomized Optimization for Efficient LLM Training
arXiv:2510. 01878v2 Announce Type: replace Abstract: Low-rank gradient optimization for large language models is currently divided into two categories: structured methods that rigorously identify subspaces, and randomized approaches employed primarily for computational efficiency.
arXiv Machine Learning
Jun 2
Riemannian Stochastic Optimization for Sufficient Dimension Reduction
arXiv:2606. 00413v1 Announce Type: cross Abstract: Sufficient dimension reduction (SDR) makes high-dimensional regression tractable by projecting the covariates onto a low-dimensional subspace that preserves the conditional mean of the response.
arXiv Machine Learning
2d ago
Solving Finite-sum Coupled Compositional Optimization via Multi-block-Single-probe Estimator
arXiv:2609.15723v1 Announce Type: new Abstract: Traditional variance reduction methods (e.g., SPIDER, SARAH, STORM) have been extensively investigated for improving the convergence rates of stochasti...
arXiv Machine Learning
Jun 10
FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection
arXiv:2604. 24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments.
arXiv Machine Learning
Jun 16
SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums
arXiv:2606. 15832v1 Announce Type: new Abstract: Empirical risk minimization on massive datasets naturally exhibits a nested double finite-sum structure, where $N=nm$ total samples are logically or physically partitioned into $n$ blocks of size $m$ (e.
arXiv Machine Learning
Jun 5
Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms
arXiv:2508. 00775v2 Announce Type: replace-cross Abstract: The design of many classical optimization algorithms is driven by the certification of linear convergence rates over classes of optimization problems.