arXiv:2602. 12471v2 Announce Type: replace Abstract: We consider the optimization problem of minimizing the logistic loss with gradient descent to train a linear model for binary classification with separable data.
By Michael Crawshaw, Mingrui Liu
arXiv:2604. 27742v2 Announce Type: replace Abstract: A fundamental dichotomy in the theory of classification sets smoothness against statistical efficiency: smooth surrogate losses such as the logistic loss enable fast $O(1/T)$ optimization but yield slow square-root $H$-consistency bounds, while piecewise-linear losses like the Hinge loss achieve optimal linear $H$-consistency rates but are non-differentiable.
By Mehryar Mohri, Yutao Zhong
The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.
By Hao Yu
The paper analyzes Gradient Descent (GD) on a minimal binary classification problem with a two‑neuron ReLU network trained on two instances. It proves that GD converges to an optimal robustness margin—maximizing the distance between the decision boundary and training points—but does so at a very slow rate, specifically θ(1/ln(t)). Empirical simulations confirm this slow convergence across various initializations, and the authors provide a rigorous analysis of GD trajectories across different activation patterns to establish the first explicit lower bound on robustness convergence in a non‑linear model.
By Guy Smorodinsky, Sveta Gimpleson, Itay Safran
arXiv:2510. 02779v4 Announce Type: replace Abstract: Recent advances have significantly improved our understanding of the generalization performance of gradient descent (GD) methods in deep neural networks.
By Yuanfan Li, Yunwen Lei, Zheng-Chu Guo, Yiming Ying
The paper introduces Curvature-Scaled Sharpness-Aware Minimization (CS‑SAM), a variant of SAM that scales both the perturbation and descent step by curvature bounds to uniformly control gradient distortion under generalized smoothness conditions. It proves deterministic ≥1/T convergence rates for nonconvex objectives, last‑iterate ≥1/T rates for convex objectives, and linear convergence under the Polyak‑Łojasiewicz condition, while recovering standard gradient descent rates when the perturbation is zero. Experiments on computer vision tasks show that CS‑SAM, particularly the (H0,H1) variant, achieves the highest mean best‑test accuracy across four settings compared to tuned SAM and USAM.
By Dimitris Oikonomou, Nicolas Loizou