arXiv Machine Learning
Jul 8
Universality of Benign Overfitting in Binary Linear Classification
arXiv:2501. 10538v3 Announce Type: replace Abstract: The practical success of deep learning has led to the discovery of several surprising phenomena.
Minimax Optimal Early-Stopped Gradient Descent for Gaussian Mixture Classification
arXiv:2608. 06250v1 Announce Type: cross Abstract: In overparameterised classification, training data can be linearly separable even when the underlying distribution is not.
Related stories
arXiv Machine Learning
Jun 16
Tight Bounds for Logistic Regression with Large Stepsize Gradient Descent in Low Dimension
arXiv:2602. 12471v2 Announce Type: replace Abstract: We consider the optimization problem of minimizing the logistic loss with gradient descent to train a linear model for binary classification with separable data.
arXiv Machine Learning
22h ago
Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixtures
arXiv:2506. 06584v2 Announce Type: replace Abstract: Learning Gaussian Mixture Models (GMMs) is a fundamental problem in statistics and machine learning, with the Expectation-Maximization (EM) algorithm and its popular variant gradient EM being arguably the most widely used algorithms in practice.
arXiv Machine Learning
Jun 19
Global Convergence of Gradient Descent for Score Matching in Gaussian Mixtures via Reverse Fisher Divergence
arXiv:2606. 19876v1 Announce Type: new Abstract: The score matching problem is a central training objective in modern generative modeling, diffusion models, fitting unnormalized statistical models, and inverse problems.
arXiv Machine Learning
Jul 27
Smart predict-then-robustly-optimize
arXiv:2607. 21773v1 Announce Type: new Abstract: In this paper, we propose and study a robust variant of the smart predict-then-optimize approach that accounts for prediction shifts due to disturbance in the covariate feature space.
arXiv Machine Learning
Jun 24
Robust and Fast Training via Per-Sample Clipping
arXiv:2605. 02701v2 Announce Type: replace-cross Abstract: We propose a robust gradient estimator based on per-sample gradient clipping and analyze its properties both theoretically and empirically.
arXiv Machine Learning
Jul 7
On Regularization via Early Stopping for Least Squares Regression
arXiv:2406. 04425v2 Announce Type: replace Abstract: A fundamental problem in machine learning is understanding the effect of early stopping on the parameters obtained and the generalization capabilities of the model.
arXiv Machine Learning
Jun 5
How abundant are good interpolators?
arXiv:2606. 06469v1 Announce Type: cross Abstract: Let $S$ be the set of unit norm linear classifiers $\theta \in \mathbb{R}^d$ which correctly classify every point of a labeled dataset $(X_i,y_i)_{i=1}^n$, $X_i \in \mathbb{R}^d$, $y_i \in \{-1,+1\}$, with a possibly negative margin $\kappa$ fixed in advance.
arXiv Machine Learning
Jul 21
Spectral-Transport Stability and Benign Overfitting in Interpolating Learning
arXiv:2604. 08625v2 Announce Type: replace-cross Abstract: We develop a theoretical framework for generalization in the interpolating regime of statistical learning.
arXiv Machine Learning
Jun 2
Fast Generalization after Interpolation via Critically Damped Momentum Optimization
arXiv:2606. 01521v1 Announce Type: new Abstract: A central problem in machine learning is that models can achieve near-perfect training performance while generalizing substantially less well to unseen examples.
arXiv Machine Learning
Jul 27
The Role of Pseudo-labels in Self-training Linear Classifiers on High-dimensional Gaussian Mixture Data
arXiv:2205. 07739v4 Announce Type: replace-cross Abstract: Self-training (ST) is a simple yet effective semi-supervised learning method.
arXiv Machine Learning
Jun 15
Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization
arXiv:2606. 13796v1 Announce Type: cross Abstract: Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution.