arXiv AI
Aug 24
Primal Acceleration of Newton's Method
arXiv:2608. 21359v1 Announce Type: cross Abstract: We develop a new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian.
Global $\mathcal{O}\left(k^{-2}\right)$ Convergence of Multilevel Gradient-Regularized Newton Methods
arXiv:2610. 35195v1 Announce Type: cross Abstract: We develop and analyze multilevel gradient-regularized Newton methods for unconstrained convex optimization with Lipschitz-continuous Hessians.
Related stories
arXiv Machine Learning
Aug 4
Simplified Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients
arXiv:2209. 03282v5 Announce Type: replace-cross Abstract: Accelerating the convergence of second-order optimization, particularly Newton-type methods, remains a pivotal challenge in algorithmic research.
arXiv Statistics ML
6d ago
Affine-invariant Cubic Newton with Weak Learners
arXiv:2610.33516v1 Announce Type: new Abstract: Modern gradient boosting algorithms are based on Newton's method, which excels empirically but lacks global convergence guarantees. While recent work o...
arXiv Machine Learning
Aug 4
Generalized Quadratic Gradient: A New Direction in Optimization via the Fusion of Positive-Definite Curvature Matrices and Gradients into A Unified Framework
arXiv:2608. 01552v1 Announce Type: cross Abstract: Quadratic Gradient (QG) is a Newton-type optimization framework that bridges first-order gradient descent and second-order optimization by incorporating curvature information into gradient updates.
arXiv Machine Learning
Aug 14
A Local-Linearly Convergent Algorithm for Nonconvex Equality-Constrained Optimization
arXiv:2608. 12665v1 Announce Type: cross Abstract: For solving nonconvex equality-constrained optimization problems, a recent Gradient-Eigenstep Algorithm by Goyens et al.
arXiv Machine Learning
Sep 10
Adaptively Incorporating Directional Hints into Zeroth-Order Optimization
arXiv:2609.08277v1 Announce Type: new Abstract: We study zeroth-order optimization of non-convex functions with the aid of directional hints, which are cheap but potentially inaccurate approximations...
arXiv Machine Learning
5d ago
Regularized Unbalanced Optimal Transport with Pearson Marginal Penalties
arXiv:2503. 15105v5 Announce Type: replace-cross Abstract: We study quadratically regularized unbalanced optimal transport with Pearson marginal penalties and a lower bound $\delta\ge0$ on the plan, including the sparse case $\delta=0$.
arXiv Machine Learning
Sep 30
Nonmonotone subgradient methods based on a local descent lemma
arXiv:2510. 19341v3 Announce Type: replace-cross Abstract: In this paper we present a nonmonotone line search subgradient algorithm tailored to upper-$\mathcal{C}^2$ functions.
Hugging Face Trending Papers
Jul 7
On the Condition Number Upper Bound of the L-BFGS Inverse Hessian Approximation Matrix with a Two-Sided Geometric Envelope Safeguarding Mechanism
The limited-memory BFGS (L-BFGS) algorithm is a cornerstone of large-scale optimization due to its linear memory and computational costs. However, in ill-conditioned or non-convex landscapes, the implicit inverse Hessian approximation can suffer from an exploding condition number, leading to numerical instability and degraded convergence.
arXiv Machine Learning
Jul 8
On the Condition Number Upper Bound of the L-BFGS Inverse Hessian Approximation Matrix with a Two-Sided Geometric Envelope Safeguarding Mechanism
arXiv:2607. 05836v1 Announce Type: cross Abstract: The limited-memory BFGS (L-BFGS) algorithm is a cornerstone of large-scale optimization due to its linear memory and computational costs.
arXiv Machine Learning
Sep 16
Near-Optimal Nonconvex Matrix Completion
arXiv:2609. 17048v1 Announce Type: cross Abstract: We study nonconvex methods for matrix completion, the problem of recovering a low-rank matrix from a subset of its entries.
arXiv Machine Learning
Oct 16
Generalization Guarantees of Randomized Block Coordinate Descent for Nonconvex Learning
The paper presents high‑probability generalization guarantees for Randomized Block Coordinate Descent (RBCD) when applied to nonconvex learning problems. It derives bounds on the gradients of both empirical and population risks under standard smoothness assumptions, and shows how polynomially decaying step sizes and early stopping can achieve optimal convergence rates for population gradients. In settings where the objective is gradient‑dominated, the authors provide functional suboptimality bounds that approach optimistic rates seen in convex or low‑noise scenarios, and they also establish dimension‑independent bounds for loss functions with a particular structure, extending applicability to high‑dimensional problems.