arXiv:2610.19652v1 Announce Type: new
Abstract: Recent work has sought to transfer first-order nonconvex optimization guarantees to zeroth-order methods through gradient estimators, typically incurri...
By Yuxuan Xia, Xiao-Tong Yuan
arXiv:2511. 19656v3 Announce Type: replace Abstract: Although upper bound guarantees for bilevel optimization have been widely studied, progress on lower bounds has been limited due to the complexity of the bilevel structure.
By Kaiyi Ji
arXiv:2606. 05438v1 Announce Type: new Abstract: We study the deterministic first-order oracle complexity of finding \(\epsilon\)-stationary points in smooth nonconvex optimization when the objective satisfies higher-order smoothness assumptions.
By Dongruo Zhou
arXiv:2610. 12328v1 Announce Type: new Abstract: We consider stochastic nonsmooth nonconvex composite optimization, which includes several important problems such as constrained optimization and the regularized training of neural networks.
By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi
arXiv:2610.27550v1 Announce Type: new
Abstract: Low curvature trace, an effective rank condition, is a common explanation for why randomized zeroth-order methods scale to high-dimensional tasks such...
By Yuxing Peng, Zhiqing Tang, Weijia Jia
arXiv:2511. 13999v2 Announce Type: replace Abstract: We study the running time, in terms of first order oracle queries, of differentially private empirical/population risk minimization of Lipschitz convex losses.
By Michael Menart, Aleksandar Nikolov
arXiv:2610. 13787v1 Announce Type: cross Abstract: We present new algorithms for stochastic optimization problems in which the objective is neither smooth nor convex, subject to a convex constraint set.
By Gon Buzaglo, Guy Kornowski
arXiv:2609. 30877v1 Announce Type: cross Abstract: We study whether the linear condition-number dependence in the stochastic complexity of SAPD+ is necessary for nonconvex-strongly-concave minimax optimization.
By Qihao Zhou
arXiv:2608. 05460v1 Announce Type: cross Abstract: This work introduces a proximal stochastic subgradient method for minimizing the sum of an expected cost, whose integrand is potentially nonsmooth and nonconvex, and a lower semicontinuous, prox-bounded function.
By Felipe Atenas, Alejandro Jofr\'e, Pedro P\'erez-Aros, David Torregrosa-Bel\'en
The paper introduces feasible‑direction methods for minimizing nonsmooth, nonconvex objectives over convex sets without using derivatives. It defines η‑feasible directions to overcome the non‑closedness of the feasible direction cone at boundary points, enabling finite‑direction search even with nonlinear constraints. Two algorithmic realizations are presented: one using an η‑direction oracle and another employing a noise‑augmented randomization scheme, both achieving finite‑time approximate d‑stationarity and establishing convergence rates under curvature and Lipschitz assumptions.
By Dan Greenstein, Nadav Hallak
arXiv:2610. 03222v1 Announce Type: cross Abstract: This paper studies the complexity of convex optimization using lazy second-order oracles (Doikov, Chayti, and Jaggi, ICML 2023), where an algorithm queries gradients every iteration and Hessians once per $m$ iterations.
By Xinliang Zhang, Lesi Chen, Chengchang Liu, Jingzhao Zhang
In this work, we study the oracle complexity of finding an $ε$-stationary point for nonconvex-strongly-convex (NC-SC) bilevel optimization using only first-order oracles. Existing methods achieving th...