arXiv Machine Learning
2d ago
Spectral Rank Certification for Foundation Model Adapters
arXiv:2608. 15351v1 Announce Type: new Abstract: Nominal LoRA rank is a design parameter; calibrated spectral evidence is a separate inferential quantity.
De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction
arXiv:2607. 16638v1 Announce Type: cross Abstract: Principal component regression (PCR) regularizes high-dimensional prediction by choosing a spectral cutoff, but rank selection cannot correct systematic inflation of the retained empirical eigenvalues.
Related stories
arXiv Machine Learning
Jun 15
The Risk Shadow of Principal Component Analysis: When 99.9999% Variance Preservation Causes Catastrophic Decision Errors
arXiv:2606. 14533v1 Announce Type: new Abstract: Principal Component Analysis (PCA) preserves variance, not the information needed to detect rare catastrophic events.
arXiv Machine Learning
Jul 27
Heavy-Tailed Principal Component Analysis
arXiv:2603. 11308v3 Announce Type: replace Abstract: Principal Component Analysis (PCA) is a cornerstone of dimensionality reduction, yet its classical formulation relies critically on second-order moments and is therefore fragile in the presence of heavy-tailed data and impulsive noise.
arXiv Machine Learning
Jul 31
Tight Sample Complexity for Low-Rank Adaptation: Matching Bounds and Rank Selection
arXiv:2607. 27680v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard mechanism for fine-tuning large pretrained models, yet its statistical properties remain only partially understood.
arXiv Machine Learning
1d ago
Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and a Tight Univariate Rate
arXiv:2608. 17573v1 Announce Type: cross Abstract: In high-dimensional online prediction, the best predictor may depend on only a few features, so regret should scale with sparsity rather than the ambient dimension.
arXiv Machine Learning
Aug 10
High-dimensional ridgeless least squares interpolation under spiked covariance structures
arXiv:2608. 07281v1 Announce Type: cross Abstract: This paper investigates the asymptotic behavior of the out-of-sample prediction risk of the high-dimensional ridgeless least-squares estimator when the feature dimension $p$ and the sample size $n$ grow proportionally.
arXiv Machine Learning
2d ago
Sufficient Dimesion Reduction via Generalized Stein's Lemma
arXiv:2608. 15121v1 Announce Type: cross Abstract: Sufficient dimension reduction (SDR) seeks the minimal subspace of the predictors that captures the full conditional distribution of the response, which is known as the central subspace (CS).
arXiv Machine Learning
Jul 28
Low-Rank Dependence Decomposition via Accelerated Symmetric Non-negative Matrix Factorization
arXiv:2607. 24518v1 Announce Type: new Abstract: Symmetric non-negative matrix factorization (SymNMF) recovers latent group structure from a dependence matrix, but its dense, quadratic-memory objective has confined prior work to moderate sizes.
arXiv Machine Learning
Jul 1
Direction-Magnitude Decomposition for Low-Rank Matrix Optimization: Faster Convergence and Saddle-to-saddle Dynamics
arXiv:2606. 31390v1 Announce Type: cross Abstract: Low-rank matrix optimization is often carried out via the Burer-Monteiro (BM) formulation, but choosing the factorization rank $r$ is delicate and can substantially slow optimization.
arXiv AI
Jul 1
When to Truncate a Feature Ranking: A Residual-Overlap Stopping Rule for Subset Selection
arXiv:2606. 31686v1 Announce Type: cross Abstract: Feature rankings are widely used in supervised feature selection because they are simple, scalable and easy to interpret.
arXiv Machine Learning
Jun 29
The Decision Geometry of Covariance Estimation for the Global Minimum-Variance Portfolio under Heavy Tails
arXiv:2606. 27462v1 Announce Type: cross Abstract: The global minimum-variance portfolio (GMVP) is the canonical decision built from an estimated covariance matrix, yet covariance estimators are universally evaluated by matrix-norm loss, which is not the object the decision depends on.
arXiv Machine Learning
Jul 7
Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation
arXiv:2303. 08777v3 Announce Type: replace-cross Abstract: Cross-validation is one of the most widely used tools for risk estimation and model selection in statistics and machine learning, yet its theoretical properties when embedded in a learning procedure remain insufficiently understood.