arXiv Statistics ML
Principal Component Regression Dominates all Monotone Spectral Filters for Linear Regression
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Jun 10
Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularization
arXiv:2509. 17251v2 Announce Type: replace-cross Abstract: Existing theory suggests that for linear regression problems categorized by capacity and source conditions, gradient descent (GD) is always minimax optimal, while both ridge regression and online stochastic gradient descent (SGD) are polynomially suboptimal for certain categories of such problems.
arXiv Machine Learning
Jul 21
De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction
arXiv:2607. 16638v1 Announce Type: cross Abstract: Principal component regression (PCR) regularizes high-dimensional prediction by choosing a spectral cutoff, but rank selection cannot correct systematic inflation of the retained empirical eigenvalues.
arXiv Machine Learning
Jul 7
On Regularization via Early Stopping for Least Squares Regression
arXiv:2406. 04425v2 Announce Type: replace Abstract: A fundamental problem in machine learning is understanding the effect of early stopping on the parameters obtained and the generalization capabilities of the model.
arXiv Machine Learning
Aug 10
High-dimensional ridgeless least squares interpolation under spiked covariance structures
arXiv:2608. 07281v1 Announce Type: cross Abstract: This paper investigates the asymptotic behavior of the out-of-sample prediction risk of the high-dimensional ridgeless least-squares estimator when the feature dimension $p$ and the sample size $n$ grow proportionally.
arXiv Machine Learning
Jul 9
Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer
arXiv:2501. 10870v2 Announce Type: replace-cross Abstract: The principal objective of this work is twofold within nonparametric regression settings: (1) to establish the minimax optimal convergence rates for fixed-bandwidth Gaussian kernel spectral algorithms when the true regression function resides in a Sobolev space, and (2) to apply Gaussian spectral algorithms for achieving robust and adaptive transfer learning under concept shift.
arXiv Machine Learning
Jun 2
Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression
arXiv:2606. 00512v1 Announce Type: new Abstract: In many modern machine learning pipelines, abundant pretrained representations serve as noisy proxy covariates, while task-specific labels remain scarce.