arXiv Machine Learning
Jun 2
Mirror Descent Under Generalized Smoothness
arXiv:2502. 00753v4 Announce Type: replace-cross Abstract: Smoothness is crucial for attaining fast rates in first-order optimization.
Fast Generalization after Interpolation via Critically Damped Momentum Optimization
arXiv:2606. 01521v1 Announce Type: new Abstract: A central problem in machine learning is that models can achieve near-perfect training performance while generalizing substantially less well to unseen examples.
Related stories
arXiv Machine Learning
Jun 5
Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity
arXiv:2503. 04712v3 Announce Type: replace-cross Abstract: We study the optimization of non-convex functions that are not necessarily smooth (gradient and/or Hessian are Lipschitz) using first order methods.
arXiv AI
Jun 2
Stability Analysis of Sharpness-Aware Minimization
arXiv:2301. 06308v2 Announce Type: replace-cross Abstract: Sharpness-aware minimization (SAM) is a training method that seeks to find flat minima in deep learning, resulting in state-of-the-art performance across various domains.
arXiv Machine Learning
Jun 30
Generalization error of min-norm interpolators in transfer learning
arXiv:2406. 13944v2 Announce Type: replace-cross Abstract: This paper establishes the generalization error of pooled min-$\ell_2$-norm interpolation in transfer learning, where data from diverse distributions are available.
arXiv Machine Learning
Jun 8
Second-Order Path Kernel Interpolation Formulas in Machine Learning
arXiv:2606. 07495v1 Announce Type: new Abstract: Understanding how training data shape neural network predictions is a central problem in modern learning theory.
arXiv AI
Jun 26
Error-Conditioned Neural Solvers
arXiv:2606. 27354v1 Announce Type: cross Abstract: Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their own constraint violations and extrapolate beyond the training distribution.
arXiv Machine Learning
Jul 15
Same Loss, Same Noise, Opposite Schedules: Noise Structure and Optimizer Normalization Jointly Determine Whether Learning-Rate Cooldown Helps
arXiv:2607. 12360v1 Announce Type: new Abstract: The cooldown phase of a warmup-stable-decay (WSD) learning-rate schedule, now a default in large-model pretraining, lowers the final training loss in some settings and does nothing in others.
arXiv Machine Learning
Jun 10
A Sketch-and-Project Analysis of Subsampled Natural Gradient Algorithms
arXiv:2508. 21022v3 Announce Type: replace Abstract: Subsampled natural gradient descent (SNG) has been used to enable high-precision scientific machine learning, but standard analyses based on stochastic preconditioning fail to provide insight into realistic small-sample settings.
arXiv Machine Learning
Jun 19
On the Oracle Complexity of Interpolation-Based Gradient Descent
arXiv:2606. 19878v1 Announce Type: new Abstract: Recent work on first-order optimizers for empirical risk minimization (ERM) has suggested that smoothness of ERM loss functions in the training data, rather than in the optimization parameters, can be leveraged to improve the oracle complexity of gradient descent (GD) methods.
Hugging Face Trending Papers
Aug 4
On the Implicit Flatness Bias of Sharpness-Aware Minimization: A Linear Stability Analysis with Quantitative Hyperparameter Bounds
Sharpness-Aware Minimization (SAM) improves generalization by seeking parameters whose loss is robust to local adversarial perturbations, but the quantitative mechanism underlying its implicit bias toward flat minima remains unclear. In particular, the perturbation radius $ρ$ is typically treated as an isolated tuning parameter, despite defining the neighborhood in which SAM measures sharpness.
arXiv Machine Learning
Jul 24
Memory-Computation Tradeoffs in Semi Amortized Parametric Optimization
arXiv:2607. 20769v1 Announce Type: new Abstract: Learning-enabled decision systems often use offline data or computation to reduce online compute cost.
arXiv Machine Learning
Jun 5
Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms
arXiv:2508. 00775v2 Announce Type: replace-cross Abstract: The design of many classical optimization algorithms is driven by the certification of linear convergence rates over classes of optimization problems.