← Back to all news
arXiv Machine Learning By Caixing Wang, Zhibo Chen, Yue Wang

Adaptive Regularization for Random Features: A Neighboring Early-Stopping Rule with Oracle-Rate Guarantees

Read the original on arXiv Machine Learning →

The paper introduces a neighboring early‑stopping rule for adaptive regularization in kernel ridge regression with random features (KRR‑RF). By using a uniform grid in inverse regularization and comparing only adjacent estimators, the method reduces discrepancy checks and can be computed directly in the random‑feature space without forming the full kernel Gram matrix. Under standard source and capacity assumptions, the selected estimator achieves the oracle polynomial learning rate up to logarithmic factors, enabling regularization selection without prior knowledge of smoothness or capacity exponents.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 27

Loop Corrections in Random Feature Models: Training Error and Generalization Gap

The paper investigates fixed‑design random feature ridge regression beyond the mean‑kernel approximation, focusing on how the predictor’s nonlinear dependence on the empirical kernel affects training error, test error, and the conditional generalization gap. By deriving covariance‑level (one‑loop) corrections via a finite resolvent identity, the authors avoid an almost‑sure Neumann‑series assumption and provide explicit remainder bounds for training error. Numerical experiments demonstrate that including mixed train–test covariance tensors is essential for accurate test error predictions and reveal a width–regularization boundary where second‑order truncation becomes unreliable.

By Taeyoung Kim
benchmarks
More like this →
arXiv AI
Jun 26

XMSE-Aware Adaptive Empirical Bayes Estimation

arXiv:2606. 26975v1 Announce Type: cross Abstract: Empirical Bayes (EB) estimators can match the first-order asymptotic risk of maximum likelihood (ML) while behaving very differently at second order: recent excess mean squared error (XMSE) analysis shows that kernel-based EB estimation may be worse than ML when the kernel is poorly aligned with the true parameter.

By Minghao Chen, Jiale Zheng
benchmarks
More like this →
arXiv Machine Learning
Oct 23

Random Projections for Spectral Algorithms: Optimal Rates in Sobolev Norms

The paper studies random projections (RP) for spectral algorithms, providing a comprehensive analysis of their generalization performance under broad conditions. By exploiting the RKHS embedding property and operator similarity, it establishes optimal learning rates in Sobolev norms that match minimax lower bounds up to logarithmic factors. For common RP implementations—randomized sketches and Nyström sub-sampling—the required projection dimension scales with the effective dimension, reducing computational cost while preserving statistical efficiency.

By Caixing Wang, Yue Wang
rag
More like this →
arXiv Machine Learning
Jul 7

Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation

arXiv:2303. 08777v3 Announce Type: replace-cross Abstract: Cross-validation is one of the most widely used tools for risk estimation and model selection in statistics and machine learning, yet its theoretical properties when embedded in a learning procedure remain insufficiently understood.

By Diego Marcondes, Cl\'audia Peixoto
safety
More like this →