arXiv Statistics ML
Hessian-Similar Optimization with Capped Participation: Lower Bounds and Virtual Masters
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Oct 5
Near-Optimal Convex Optimization with Lazy Second-Order Oracles
arXiv:2610. 03222v1 Announce Type: cross Abstract: This paper studies the complexity of convex optimization using lazy second-order oracles (Doikov, Chayti, and Jaggi, ICML 2023), where an algorithm queries gradients every iteration and Hessians once per $m$ iterations.
arXiv Machine Learning
Jul 2
Fast Score-Based Sampling via Log-Concave Reductions
arXiv:2512. 24152v2 Announce Type: replace-cross Abstract: Sampling based on score diffusions has led to striking empirical results, and has attracted considerable attention from various research communities.
arXiv Machine Learning
Jun 19
Beyond Averaging in John Ellipsoid Approximation: High-Accuracy Algorithms in the Leverage-Score Model
arXiv:2606. 20082v1 Announce Type: cross Abstract: The John ellipsoid of a symmetric polytope $P=\{\mathbf{x}\in\mathbb{R}^d:\|\mathbf{A}\mathbf{x}\|_\infty\le1\}$, $\mathbf{A}\in\mathbb{R}^{n\times d}$, is computed by a long line of leverage-score algorithms, from Cohen, Cousins, Lee and Yang (COLT 2019) to its successors [WY24, CLS+25], all reaching a $(1+\varepsilon)$-approximation in $\Theta(\varepsilon^{-1}\log(n/d))$ iterations.
arXiv Machine Learning
Sep 2
Dense Weak Hiding: Closing Complexity Gaps in Nonconvex and PL Finite-Sum Optimization under Individual Smoothness
The paper establishes the optimal incremental first‑order oracle (IFO) complexity for nonconvex finite‑sum optimization under individual smoothness, proving a matching lower bound that closes a previously missing √{n} factor. It also refines the analysis of the PAGE algorithm under the global Polyak‑Lojasiewicz condition, providing tighter guarantees for different ranges of the condition number. The authors introduce a novel dense weak hiding construction that yields these lower bounds and demonstrates the limits of existing methods.
arXiv Machine Learning
Oct 21
Lower Bounds for Smooth Nonconvex Zeroth-Order Optimization: The Price of Determinism under Low Curvature Trace
arXiv:2610.27550v1 Announce Type: new Abstract: Low curvature trace, an effective rank condition, is a common explanation for why randomized zeroth-order methods scale to high-dimensional tasks such...
arXiv Machine Learning
Oct 19
Guarded Block Transfers for Polytope Distance: Linear Convergence with Explicit Constants and Gap-Certified Screening for SVM Training
arXiv:2610.15829v2 Announce Type: replace Abstract: Hard-margin SVM training is the distance problem between two convex hulls, which the Triangle Algorithm (TA) solves with a certified duality gap. W...