arXiv Statistics ML
Affine-invariant Cubic Newton with Weak Learners
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Aug 4
Simplified Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients
arXiv:2209. 03282v5 Announce Type: replace-cross Abstract: Accelerating the convergence of second-order optimization, particularly Newton-type methods, remains a pivotal challenge in algorithmic research.
arXiv Machine Learning
Aug 26
Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence
arXiv:2608.22129v2 Announce Type: replace Abstract: Cubic regularized Newton methods have the optimal $\mathcal{O}(\epsilon^{-3/2})$ global rate, but a dense subproblem solve limits the feasible bloc...
arXiv Machine Learning
Jul 28
Learning from the Descent Direction: Adaptive Gradient Descent under One-Sided H\"older Regularity
arXiv:2607. 22906v1 Announce Type: new Abstract: We study adaptive gradient descent for continuously differentiable, possibly nonconvex objectives under one-sided H\"older regularity.
arXiv AI
Aug 24
Primal Acceleration of Newton's Method
arXiv:2608. 21359v1 Announce Type: cross Abstract: We develop a new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian.
arXiv Machine Learning
Jun 2
Mirror Descent Under Generalized Smoothness
arXiv:2502. 00753v4 Announce Type: replace-cross Abstract: Smoothness is crucial for attaining fast rates in first-order optimization.
arXiv Machine Learning
Oct 22
Curvature-Scaled Sharpness-Aware Minimization: Convergence Beyond Smoothness
The paper introduces Curvature-Scaled Sharpness-Aware Minimization (CS‑SAM), a variant of SAM that scales both the perturbation and descent step by curvature bounds to uniformly control gradient distortion under generalized smoothness conditions. It proves deterministic ≥1/T convergence rates for nonconvex objectives, last‑iterate ≥1/T rates for convex objectives, and linear convergence under the Polyak‑Łojasiewicz condition, while recovering standard gradient descent rates when the perturbation is zero. Experiments on computer vision tasks show that CS‑SAM, particularly the (H0,H1) variant, achieves the highest mean best‑test accuracy across four settings compared to tuned SAM and USAM.