Unit excitation (UE) is a common assumption in stochastic saddle avoidance: the stochastic error must have a uniformly positive component along every direction, in expectation. This condition gives a direct way to rule out convergence to strict saddles, but it also oversimplifies the actual noise structure, and does not match many stochastic optimization regimes.
arXiv:2608. 05460v1 Announce Type: cross Abstract: This work introduces a proximal stochastic subgradient method for minimizing the sum of an expected cost, whose integrand is potentially nonsmooth and nonconvex, and a lower semicontinuous, prox-bounded function.
By Felipe Atenas, Alejandro Jofr\'e, Pedro P\'erez-Aros, David Torregrosa-Bel\'en
arXiv:2505.20817v3 Announce Type: replace-cross
Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...
By Taha El Bakkali El Kadi, Savelii Chezhegov, Aleksandr Beznosikov, Samuel Horv\'ath, Eduard Gorbunov
arXiv:2606. 00520v1 Announce Type: cross Abstract: Many stochastic gradient methods are believed not to converge when the noise in stochastic gradients has only a finite $p$-th moment for $p\in\left(1,2\right)$, a setting known as the heavy-tailed noise assumption.
By Zijian Liu
arXiv:2606. 03769v1 Announce Type: cross Abstract: We study the robustness of stochastic mirror descent (SMD) under heavy-tailed noise, focusing on whether the method retains its convergence guarantees when run with infinite-variance stochastic gradient input.
By Pierre-Louis Cauvin, Panayotis Mertikopoulos
arXiv:2510. 10697v2 Announce Type: replace-cross Abstract: We define a stochastic variant of the proximal point algorithm in the general setting of nonlinear Hadamard spaces for approximating zeros of the mean of a stochastically perturbed monotone vector field.
By Nicholas Pischke
arXiv:2607. 09097v1 Announce Type: cross Abstract: We study stochastic fixed-point equations $\mathbf{T}(\mathbf{x}) = \mathbf{x}$ over normed spaces $(\mathcal{E}, \|\cdot\|)$, where the operator $\mathbf{T}$ is nonexpansive or contractive and is accessed only through unbiased stochastic evaluations with bounded second central moment.
By Jelena Diakonikolas
The paper investigates Polyak-type step-size strategies for extragradient methods applied to deterministic and stochastic monotone root-finding problems. It shows that the projection-based correction in deterministic extragradient can be derived by minimizing an upper bound on the distance to a solution, mirroring classical Polyak step-size construction. The authors provide a unified deterministic analysis that does not require global Lipschitz continuity, achieving sublinear convergence under H"older or “(L0, L1)-Lipschitz” conditions and linear convergence with strong monotonicity, and extend the approach to stochastic settings with both direct and decreasing step-size variants.
By TaeHo Yoon, Sayantan Choudhury, Ezra Greenberg, Nicolas Loizou
arXiv:2607. 15412v1 Announce Type: new Abstract: Multi-objective learning (MOL) aims to optimize multiple objectives simultaneously.
By Chentong Huang, Lisha Chen
arXiv:2502.21099v3 Announce Type: replace-cross
Abstract: This paper proposes {\sf AEPG-SPIDER}, an Adaptive Extrapolated Proximal Gradient (AEPG) method with variance reduction for minimizing compos...
By Ganzhao Yuan
arXiv:2604. 08580v2 Announce Type: replace-cross Abstract: Reward fine-tuning of diffusion and flow models and sampling from tilted or Boltzmann distributions can both be formulated as stochastic optimal control (SOC) problems, where learning an optimal generative dynamics corresponds to optimizing a control under SDE constraints.
By Carles Domingo-Enrich, Jiequn Han
The paper introduces single-loop stochastic projected damped extragradient (SPDE) and its variance-reduced variant (VR-SPDE) for stochastic nonconvex–(strongly) concave minimax problems. It provides SFO complexity bounds for achieving game stationarity and optimization stationarity, improving upon previous multi-loop methods while maintaining a single-loop structure. The results claim the best-known SFO complexities for these stationarity criteria among single-loop stochastic first‑order methods.
By Huiling Zhang, Minhao Zhang, Zi Xu