arXiv Machine Learning
Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jul 7
Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation
arXiv:2303. 08777v3 Announce Type: replace-cross Abstract: Cross-validation is one of the most widely used tools for risk estimation and model selection in statistics and machine learning, yet its theoretical properties when embedded in a learning procedure remain insufficiently understood.
arXiv Machine Learning
Jun 30
Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model
arXiv:2606. 28573v1 Announce Type: new Abstract: Modern machine learning models are trained by optimizing high-dimensional non-convex empirical risk functions.
arXiv Machine Learning
Jul 3
On the Asymptotics of Self-Supervised Pre-training: Two-Stage M-Estimation and Representation Symmetry
arXiv:2603. 27631v2 Announce Type: replace Abstract: Self-supervised pre-training, where large corpora of unlabeled data are used to learn representations for downstream fine-tuning, has become a cornerstone of modern machine learning.
Hugging Face Trending Papers
Oct 15
Hard Way, The Only Way: Provable Merits of Training on Hardest Examples
Despite a surge of empirical interest in the ``unreasonable effectiveness'' of drastic data pruning for AI systems, a unified theoretical understanding of this phenomenon remains elusive. In this work...
arXiv Machine Learning
Jul 27
Smart predict-then-robustly-optimize
arXiv:2607. 21773v1 Announce Type: new Abstract: In this paper, we propose and study a robust variant of the smart predict-then-optimize approach that accounts for prediction shifts due to disturbance in the covariate feature space.
arXiv Machine Learning
Oct 12
Fast Rates for Nonstationary Weighted Risk Minimization
The paper investigates weighted empirical risk minimization under nonstationary data, decomposing the excess risk into a learning component and a drift component. It establishes oracle inequalities for the learning error under mixing conditions, uniformly over arbitrary weight classes, and derives explicit bounds for drift errors in square loss settings, covering structural‑break and smoothly drifting processes. The analysis is extended to auto‑regression problems with linear models, basis approximations, and neural networks, revealing that optimal weights balance learning and drift via a bias–variance trade‑off and recover minimax‑optimal rates in unweighted, stationary cases.