arXiv Machine Learning
Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysis
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jul 28
Risk reversal for least squares estimators under nested convex constraints
arXiv:2601. 16041v2 Announce Type: replace-cross Abstract: In constrained stochastic optimization, one expects that restricting the feasible set, provided it still contains the true parameter, should not increase the statistical risk of the corresponding projection estimator.
arXiv Machine Learning
Jun 30
Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model
arXiv:2606. 28573v1 Announce Type: new Abstract: Modern machine learning models are trained by optimizing high-dimensional non-convex empirical risk functions.
arXiv Machine Learning
4d ago
High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness
arXiv:2505.20817v3 Announce Type: replace-cross Abstract: Gradient clipping is widely used in language-model training to control heavy-tailed gradient noise and can improve convergence guarantees ove...
arXiv Machine Learning
Jun 25
A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization
arXiv:2602. 02877v2 Announce Type: replace Abstract: This paper studies optimization for a family of problems termed $\textbf{compositional entropic risk minimization}$, in which each data's loss is formulated as a Log-Expectation-Exponential (Log-E-Exp) function.
arXiv Machine Learning
Aug 27
Beyond Optimal Rates in Stochastic Optimization: Trajectory-Adaptive Stopping Rules
arXiv:2608. 25551v1 Announce Type: new Abstract: Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory.
arXiv Machine Learning
Jul 13
Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization
arXiv:2405. 00914v4 Announce Type: replace-cross Abstract: We present in this paper novel accelerated fully first-order methods in \emph{Bilevel Optimization} (BLO).