arXiv Machine Learning

The Exponential Price of Determinism in Nonsmooth Nonconvex Optimization

arXiv:2609. 23837v1 Announce Type: cross Abstract: We study the complexity of finding $(\delta,\epsilon)$-Goldstein stationary points of nonsmooth nonconvex Lipschitz functions.

arXiv Machine Learning
Aug 10

A proximal subgradient method for nonconvex stochastic optimization under the Kurdyka-{\L}ojasiewicz condition

arXiv:2608. 05460v1 Announce Type: cross Abstract: This work introduces a proximal stochastic subgradient method for minimizing the sum of an expected cost, whose integrand is potentially nonsmooth and nonconvex, and a lower semicontinuous, prox-bounded function.

By Felipe Atenas, Alejandro Jofr\'e, Pedro P\'erez-Aros, David Torregrosa-Bel\'en
arXiv Machine Learning
Nov 6

Randomized and Oracle-Based Feasible Directions Methods for Nonsmooth Nonconvex Constrained Optimization

The paper introduces feasible‑direction methods for minimizing nonsmooth, nonconvex objectives over convex sets without using derivatives. It defines η‑feasible directions to overcome the non‑closedness of the feasible direction cone at boundary points, enabling finite‑direction search even with nonlinear constraints. Two algorithmic realizations are presented: one using an η‑direction oracle and another employing a noise‑augmented randomization scheme, both achieving finite‑time approximate d‑stationarity and establishing convergence rates under curvature and Lipschitz assumptions.

By Dan Greenstein, Nadav Hallak
arXiv Machine Learning
Oct 5

Near-Optimal Convex Optimization with Lazy Second-Order Oracles

arXiv:2610. 03222v1 Announce Type: cross Abstract: This paper studies the complexity of convex optimization using lazy second-order oracles (Doikov, Chayti, and Jaggi, ICML 2023), where an algorithm queries gradients every iteration and Hessians once per $m$ iterations.

By Xinliang Zhang, Lesi Chen, Chengchang Liu, Jingzhao Zhang