arXiv Machine Learning By Dan Greenstein, Nadav Hallak

Randomized and Oracle-Based Feasible Directions Methods for Nonsmooth Nonconvex Constrained Optimization

Read the original on arXiv Machine Learning →

The paper introduces feasible‑direction methods for minimizing nonsmooth, nonconvex objectives over convex sets without using derivatives. It defines η‑feasible directions to overcome the non‑closedness of the feasible direction cone at boundary points, enabling finite‑direction search even with nonlinear constraints. Two algorithmic realizations are presented: one using an η‑direction oracle and another employing a noise‑augmented randomization scheme, both achieving finite‑time approximate d‑stationarity and establishing convergence rates under curvature and Lipschitz assumptions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Jul 9

Geometric--Nongeometric Optimizer Calculus: A Modular Language for Reachable Gradient Methods

arXiv:2607. 07206v1 Announce Type: new Abstract: Adaptive optimizers mix several mechanisms: a metric or preconditioner maps gradients to descent directions, while estimation, memory, step-size control, constraints, stochasticity, target modification, and discretization determine which directions are available and how they are used.

By Zavier Li
arXiv Machine Learning
Jul 23

Online Optimization of Difference-of-Convex Compositions with Smooth Mappings

arXiv:2607. 19553v1 Announce Type: cross Abstract: We study online optimization for a broad class of structured non-convex non-smooth problems where each loss is a composition of a difference-of-convex function with a smooth mapping, and the feasible region is defined by constraint functions of the same kind.

By Jingwei Ji, Jong-Shi Pang, Renyuan Xu