arXiv Machine Learning June 10, 2026 By Jingfeng Wu, Peter L. Bartlett, Sham M. Kakade, Jason D. Lee, Bin Yu

Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularization

Read the original on arXiv Machine Learning →

arXiv:2509. 17251v2 Announce Type: replace-cross Abstract: Existing theory suggests that for linear regression problems categorized by capacity and source conditions, gradient descent (GD) is always minimax optimal, while both ridge regression and online stochastic gradient descent (SGD) are polynomially suboptimal for certain categories of such problems.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 7

On Regularization via Early Stopping for Least Squares Regression

arXiv:2406. 04425v2 Announce Type: replace Abstract: A fundamental problem in machine learning is understanding the effect of early stopping on the parameters obtained and the generalization capabilities of the model.

By Rishi Sonthalia, Jackie Lok, Elizaveta Rebrova

arXiv Machine Learning

Aug 4

A Simple Approximation to the Distribution of the Ridge Regression Estimator

arXiv:2608. 02539v1 Announce Type: cross Abstract: We present a simple Gaussian approximation to the finite-sample distribution of the classical ridge regression estimator.

By Jos\'e Luis Montiel Olea, Ryan Strong, Amilcar Velez, Zhuoheng Xu, Haomin Yu

safety

arXiv Machine Learning

Jun 19

Benign overfitting beyond prediction: The ordinary least squares interpolator

arXiv:2309. 15769v3 Announce Type: replace-cross Abstract: Recent advances in deep learning have highlighted the phenomenon of benign overfitting in overparameterized statistical models, sparking significant interest in understanding its foundations.

By Dennis Shen, Dogyoon Song, Peng Ding, Jasjeet S. Sekhon

safety

arXiv Machine Learning

Jun 2

Optimal Regularization for Performative Learning

arXiv:2510. 12249v2 Announce Type: replace Abstract: In performative learning, the data distribution reacts to the deployed model - for example, because strategic users adapt their features to game it - which creates a more complex dynamic than in classical supervised learning.

By Edwige Cyffers, Alireza Mirrokni, Marco Mondelli

arXiv Machine Learning

Jul 2

Homogenization of $\ell_2$-Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent

arXiv:2607. 00207v1 Announce Type: cross Abstract: We develop a framework for analyzing the learning dynamics of $\ell_2$-adversarial training of single-index models on Gaussian mixtures in the high-dimensional limit under streaming stochastic gradient descent (SGD).

By Fabrizzio Sabelli

safety

arXiv Machine Learning

Jul 17

To Grok Grokking: Provable Grokking in Ridge Regression

arXiv:2601. 19791v4 Announce Type: replace Abstract: We study grokking, the onset of generalization long after overfitting, in a classical ridge regression setting.

By Mingyue Xu, Gal Vardi, Itay Safran

Related stories

On Regularization via Early Stopping for Least Squares Regression

A Simple Approximation to the Distribution of the Ridge Regression Estimator

Benign overfitting beyond prediction: The ordinary least squares interpolator

Optimal Regularization for Performative Learning

Homogenization of $\ell_2$-Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent

To Grok Grokking: Provable Grokking in Ridge Regression