arXiv:2610.07290v1 Announce Type: cross Abstract: Recent advances in penalty-based methods for stochastic bilevel optimization (SBO) have eliminated the need for second-order derivative oracles. Howe...
arXiv:2608. 12665v1 Announce Type: cross Abstract: For solving nonconvex equality-constrained optimization problems, a recent Gradient-Eigenstep Algorithm by Goyens et al.
arXiv:2608. 05460v1 Announce Type: cross Abstract: This work introduces a proximal stochastic subgradient method for minimizing the sum of an expected cost, whose integrand is potentially nonsmooth and nonconvex, and a lower semicontinuous, prox-bounded function.
arXiv:2508.16791v2 Announce Type: replace-cross Abstract: We develop a novel past-extragradient-type algorithmic framework, combining both Nesterov's \textit{acceleration} and \textit{variance-reduct...
The paper investigates variance reduction techniques for non‑convex optimization when gradient evaluations are delayed. It introduces an ordered recursive momentum estimator whose oracle complexity depends on smoothness, variance, and delay, and provides bounds for both stochastic and finite‑sum settings. An asynchronous snapshot variant is also proposed, allowing updates during full‑gradient computations under a completion‑time bound.
arXiv:2609.15723v1 Announce Type: new Abstract: Traditional variance reduction methods (e.g., SPIDER, SARAH, STORM) have been extensively investigated for improving the convergence rates of stochasti...
arXiv:2608. 12009v1 Announce Type: cross Abstract: Bregman proximal stochastic gradient (BPSG) methods bring variance-reduced composite optimization to objectives whose geometry is poorly captured by Euclidean smoothness.
The paper introduces a parallel architecture for stochastic gradient methods that adaptively selects the number of iterations. An algorithm A(x₀, y) takes an initial point and a step limit y, and p processors search for an appropriate iteration count T using a prescribed function h. The framework guarantees a (p, αₚ)-approximation, meaning for any T ≥ T₀ there exists a processor and stage where the cumulative iterations lie within a factor αₚ of T, and the authors prove tight lower bounds for αₚ while presenting simple arithmetic stochastic gradient methods that use only divisions by powers of two.
arXiv:2608. 12043v1 Announce Type: cross Abstract: Acceleration for deterministic root-finding problems has been extensively studied in recent years; specifically, the anchor-based, or Halpern-type methods achieve optimal convergence rates with respect to the operator norm.
The paper investigates Polyak-type step-size strategies for extragradient methods applied to deterministic and stochastic monotone root-finding problems. It shows that the projection-based correction in deterministic extragradient can be derived by minimizing an upper bound on the distance to a solution, mirroring classical Polyak step-size construction. The authors provide a unified deterministic analysis that does not require global Lipschitz continuity, achieving sublinear convergence under H"older or “(L0, L1)-Lipschitz” conditions and linear convergence with strong monotonicity, and extend the approach to stochastic settings with both direct and decreasing step-size variants.
arXiv:2406.02413v4 Announce Type: replace-cross Abstract: We propose a new class of fast Krasnoselkii--Mann methods with variance reduction to solve a finite-sum co-coercive equation $Gx = 0$. Our al...
arXiv:2603.15576v2 Announce Type: replace-cross Abstract: This paper develops new variance-reduction techniques for the forward-reflected-backward splitting (FRBS) method to solve a class of possibly...