arXiv:2608. 07281v1 Announce Type: cross Abstract: This paper investigates the asymptotic behavior of the out-of-sample prediction risk of the high-dimensional ridgeless least-squares estimator when the feature dimension $p$ and the sample size $n$ grow proportionally.
arXiv:2309. 15769v3 Announce Type: replace-cross Abstract: Recent advances in deep learning have highlighted the phenomenon of benign overfitting in overparameterized statistical models, sparking significant interest in understanding its foundations.
arXiv:2608. 02539v1 Announce Type: cross Abstract: We present a simple Gaussian approximation to the finite-sample distribution of the classical ridge regression estimator.
arXiv:2609.38011v1 Announce Type: new Abstract: Modern machine learning systems are trained on mixtures of data from different domains, and choosing the right mixture can substantially improve downst...
arXiv:2608. 28564v1 Announce Type: cross Abstract: We study kernel ridge regression under anisotropic Gaussian data, where the input covariance decays as a power law with exponent $\alpha\geq 0$ for polynomial inner-product kernels.
The paper examines whether benign overfitting—where highly overparameterized models still predict well—occurs in equity return prediction. It finds a double‑descent risk curve for ridgeless models and shows that while ridge regularization slightly improves performance, the advantage vanishes at high parameter‑to‑observation ratios. Ultimately, both models fail to beat a simple historical average, indicating that standard equity predictors lack genuine forecasting power even with flexible machine learning methods.
arXiv:2509. 17251v2 Announce Type: replace-cross Abstract: Existing theory suggests that for linear regression problems categorized by capacity and source conditions, gradient descent (GD) is always minimax optimal, while both ridge regression and online stochastic gradient descent (SGD) are polynomially suboptimal for certain categories of such problems.
arXiv:2608. 01032v1 Announce Type: new Abstract: Training error is what we can observe on a training set; test error is the quantity we actually care about.
The paper develops a statistical theory for minimum‑norm interpolation in high‑dimensional regression, showing how regularization geometry and signal sparsity affect generalization. It identifies regimes where sparsity‑promoting regularizers yield exact interpolation that is far more accurate than approximate fitting, and proves a zero–one generalization law for strongly overparameterized noiseless problems. The authors also characterize training and generalization errors along ρ‑regularization paths when feature dimension and sample size are proportional, demonstrating that generalization improves with more sparsity‑promoting norms and sparser targets, and that small changes in regularization strength can cause large shifts in generalization. whyItMatters":"The work provides a quantitative understanding of delayed generalization (grokking) and reveals a statistical instability in minimum‑norm interpolation, offering insights that could guide the design of regularizers for better generalization in overparameterized models."
arXiv:2604. 08625v2 Announce Type: replace-cross Abstract: We develop a theoretical framework for generalization in the interpolating regime of statistical learning.
arXiv:2607. 24041v1 Announce Type: cross Abstract: Over-parameterized linear regression has been widely studied over the last decade.
arXiv:2109.02355v2 Announce Type: replace Abstract: The last decade of progress in machine learning (ML), especially the deep learning era, has raised a number of scientific questions that challenge...
arXiv:2609.39440v1 Announce Type: new Abstract: We compare the instance-wise, finite-sample risks of monotone spectral filters for linear regression, a broad class of estimators including principal c...