The paper studies stochastic multi‑level optimization where the objective is a nested composition of smooth non‑convex functions. It introduces momentum‑based estimators that track function values at each level, achieving an optimal sample complexity of ≠(ε⁻⁴) for finding an ε‑stationary point without relying on average smoothness assumptions. The authors also present a batch‑free variant using first‑order approximations and clipping, and demonstrate the methods on risk‑averse portfolio optimization and hierarchical tilted empirical risk minimization.
arXiv:2609.15679v1 Announce Type: cross Abstract: This paper studies projection-free algorithms for stochastic constrained multi-level compositional optimization. In this context, the objective funct...
arXiv:2406. 13041v3 Announce Type: replace Abstract: Lower-bound analyses for nonconvex strongly-concave minimax optimization problems have shown that stochastic first-order algorithms require at least $\mathcal{O}(\varepsilon^{-4})$ sample complexity to find an $\varepsilon$-stationary point.
arXiv:2607. 08104v1 Announce Type: new Abstract: Stochastic gradient descent (SGD) is a cornerstone of modern optimization.
arXiv:2510.11676v2 Announce Type: replace-cross Abstract: We study convex composite optimization problems, where the objective function is given by the sum of a prox-friendly function and a convex fu...
arXiv:2607. 03871v1 Announce Type: new Abstract: Minimum maximum mean discrepancy (MMD) estimation has emerged as a robust and likelihood-free alternative to maximum likelihood estimation for parameter estimation.
arXiv:2609.15723v1 Announce Type: new Abstract: Traditional variance reduction methods (e.g., SPIDER, SARAH, STORM) have been extensively investigated for improving the convergence rates of stochasti...
arXiv:2608. 12009v1 Announce Type: cross Abstract: Bregman proximal stochastic gradient (BPSG) methods bring variance-reduced composite optimization to objectives whose geometry is poorly captured by Euclidean smoothness.
arXiv:2505. 01258v2 Announce Type: replace-cross Abstract: Bilevel optimization has recently attracted significant attention in machine learning due to its wide range of applications and advanced hierarchical optimization capabilities.
arXiv:2605. 26000v2 Announce Type: replace-cross Abstract: Stochastic gradient descent (SGD) is foundational to large-scale statistical learning and stochastic optimization.
arXiv:2605.28517v2 Announce Type: replace-cross Abstract: Stochastic gradient descent with momentum (SGDM) is one of the most widely used optimization algorithms in machine learning. While optimizati...
The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.
arXiv:2607. 14731v1 Announce Type: new Abstract: Local SGD, also known as Federated Averaging, is a widely used distributed optimization algorithm.