arXiv Machine Learning
Jun 2
Mirror Descent Under Generalized Smoothness
arXiv:2502. 00753v4 Announce Type: replace-cross Abstract: Smoothness is crucial for attaining fast rates in first-order optimization.
Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity
arXiv:2503. 04712v3 Announce Type: replace-cross Abstract: We study the optimization of non-convex functions that are not necessarily smooth (gradient and/or Hessian are Lipschitz) using first order methods.
Related stories
arXiv Machine Learning
Aug 4
Simplified Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients
arXiv:2209. 03282v5 Announce Type: replace-cross Abstract: Accelerating the convergence of second-order optimization, particularly Newton-type methods, remains a pivotal challenge in algorithmic research.
arXiv Machine Learning
Jun 2
Fast Generalization after Interpolation via Critically Damped Momentum Optimization
arXiv:2606. 01521v1 Announce Type: new Abstract: A central problem in machine learning is that models can achieve near-perfect training performance while generalizing substantially less well to unseen examples.
arXiv Machine Learning
Jul 23
On Optimization Complexity of Second-Order Certified Unlearning
arXiv:2607. 20192v1 Announce Type: new Abstract: We study machine unlearning: the removal of memorized training data from a trained model.
Hugging Face Trending Papers
Jul 22
On Optimization Complexity of Second-Order Certified Unlearning
We study machine unlearning: the removal of memorized training data from a trained model. Specifically, we investigate the algorithmic complexity of certified unlearning from an optimization perspective.
arXiv Machine Learning
Jun 16
A Fully First-Order Layer for Differentiable Optimization
arXiv:2512. 02494v2 Announce Type: replace Abstract: Differentiable optimization layers enable learning systems to make decisions by solving embedded optimization problems.
arXiv Machine Learning
Jun 24
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum
arXiv:2406. 13041v3 Announce Type: replace Abstract: Lower-bound analyses for nonconvex strongly-concave minimax optimization problems have shown that stochastic first-order algorithms require at least $\mathcal{O}(\varepsilon^{-4})$ sample complexity to find an $\varepsilon$-stationary point.
arXiv Machine Learning
Jul 17
What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity
arXiv:2607. 14731v1 Announce Type: new Abstract: Local SGD, also known as Federated Averaging, is a widely used distributed optimization algorithm.
arXiv Machine Learning
Jun 5
Sharp First-Order Lower Bounds for Higher-Order Smooth Nonconvex Optimization
arXiv:2606. 05438v1 Announce Type: new Abstract: We study the deterministic first-order oracle complexity of finding \(\epsilon\)-stationary points in smooth nonconvex optimization when the objective satisfies higher-order smoothness assumptions.
arXiv AI
Jun 2
Stability Analysis of Sharpness-Aware Minimization
arXiv:2301. 06308v2 Announce Type: replace-cross Abstract: Sharpness-aware minimization (SAM) is a training method that seeks to find flat minima in deep learning, resulting in state-of-the-art performance across various domains.
arXiv AI
Jun 24
Minimisation of Quasar-Convex Functions Using Random Zeroth-Order Oracles
arXiv:2505. 02281v3 Announce Type: replace-cross Abstract: This paper explores the performance of a random Gaussian smoothing zeroth-order (ZO) scheme for minimising quasar-convex (QC) and strongly quasar-convex (SQC) functions in both unconstrained and constrained settings.
arXiv Machine Learning
Jun 5
Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms
arXiv:2508. 00775v2 Announce Type: replace-cross Abstract: The design of many classical optimization algorithms is driven by the certification of linear convergence rates over classes of optimization problems.