The paper investigates sparse data augmentation for nonconvex optimization in geometric machine learning. It shows that using a small, fixed sample of transformations—obtained before optimization—allows gradient descent to achieve an ε‑stationary point of the fully augmented objective with ≤ O((log|G|+log(1/δ))/ε²) transformation queries. This is more efficient than both full augmentation and standard group‑SGD, which require O(1/ε⁴) queries.
arXiv:2606. 24418v1 Announce Type: new Abstract: Data augmentation is a simple and model-agnostic approach for exploiting known invariances in learning problems.
By Behrooz Tahmasebi, Melanie Weber, Stefanie Jegelka
arXiv:2606. 15832v1 Announce Type: new Abstract: Empirical risk minimization on massive datasets naturally exhibits a nested double finite-sum structure, where $N=nm$ total samples are logically or physically partitioned into $n$ blocks of size $m$ (e.
By Igor Sokolov, Laurent Condat, Peter Richt\'arik
arXiv:2601. 18115v2 Announce Type: replace Abstract: We study the problem of learning a single neuron under standard squared loss in the presence of arbitrary label noise and group-level distributional shifts, for a broad family of covariate distributions.
By Guyang Cao, Shuyao Li, Sushrut Karmalkar, Jelena Diakonikolas
arXiv:2504.09409v3 Announce Type: replace-cross
Abstract: In this paper, we study nonconvex constrained stochastic zeroth-order optimization problems with exact constraints and stochastic objective e...
By Qiankun Shi, Han Yuan, Xiao Wang, Hao Wang
The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.
By Feng Zhu, Robert W. Heath Jr., Aritra Mitra
The paper studies algorithms for computing the Entropic Gromov-Wasserstein (EGW) distance, a measure of discrepancy between metric measure spaces. It introduces Averaged Mirror Descent (AMD), which averages successive Mirror Descent steps and is proven to converge for any cost function, and shows that a dual gradient method with a fixed step size also converges for arbitrary costs, even when iterations are inexact. Empirical comparisons demonstrate that both AMD and the dual gradient method succeed on cases where classical Mirror Descent fails.
By Joanna Marks, Gabriel Rioux, Riccardo Passeggeri
arXiv:2602. 02877v2 Announce Type: replace Abstract: This paper studies optimization for a family of problems termed $\textbf{compositional entropic risk minimization}$, in which each data's loss is formulated as a Log-Expectation-Exponential (Log-E-Exp) function.
By Xiyuan Wei, Linli Zhou, Bokun Wang, Chih-Jen Lin, Tianbao Yang
arXiv:2511. 13592v2 Announce Type: replace-cross Abstract: The existing method of GS-PowerOpt solves the non-convex optimization problem of the form $\max_{\boldsymbol{x} \in \mathbb{R}^d} f(\boldsymbol{x})$ through maximizing a Gaussian-smoothed surrogate $F_{N,\sigma}(\boldsymbol{\mu}) = \mathbb{E}_{\boldsymbol{x}\sim\mathcal{N}(\boldsymbol{\mu},\sigma^2 I_d)}[e^{N f(\boldsymbol{x})}]$.
By Chen Xu
arXiv:2602. 02431v2 Announce Type: replace-cross Abstract: It is folklore that reusing training data more than once can improve the statistical efficiency of gradient-based learning.
By Filip Kova\v{c}evi\'c, Hong Chang Ji, Denny Wu, Mahdi Soltanolkotabi, Marco Mondelli
arXiv:2310. 15976v4 Announce Type: replace Abstract: signSGD is attractive in nonconvex optimization because it communicates sign-valued rather than full-precision gradients.
By Zhen Qin, Zhishuai Liu, Pan Xu
arXiv:2505.20817v3 Announce Type: replace-cross
Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...
By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov