arXiv:2303. 08777v3 Announce Type: replace-cross Abstract: Cross-validation is one of the most widely used tools for risk estimation and model selection in statistics and machine learning, yet its theoretical properties when embedded in a learning procedure remain insufficiently understood.
arXiv:2606. 28573v1 Announce Type: new Abstract: Modern machine learning models are trained by optimizing high-dimensional non-convex empirical risk functions.
arXiv:2603. 27631v2 Announce Type: replace Abstract: Self-supervised pre-training, where large corpora of unlabeled data are used to learn representations for downstream fine-tuning, has become a cornerstone of modern machine learning.
Despite a surge of empirical interest in the ``unreasonable effectiveness'' of drastic data pruning for AI systems, a unified theoretical understanding of this phenomenon remains elusive. In this work...
arXiv:2607. 21773v1 Announce Type: new Abstract: In this paper, we propose and study a robust variant of the smart predict-then-optimize approach that accounts for prediction shifts due to disturbance in the covariate feature space.
The paper investigates weighted empirical risk minimization under nonstationary data, decomposing the excess risk into a learning component and a drift component. It establishes oracle inequalities for the learning error under mixing conditions, uniformly over arbitrary weight classes, and derives explicit bounds for drift errors in square loss settings, covering structural‑break and smoothly drifting processes. The analysis is extended to auto‑regression problems with linear models, basis approximations, and neural networks, revealing that optimal weights balance learning and drift via a bias–variance trade‑off and recover minimax‑optimal rates in unweighted, stationary cases.
arXiv:2604. 03146v2 Announce Type: replace-cross Abstract: We study high-dimensional convex empirical risk minimization (ERM) under general non-Gaussian data designs.
arXiv:2603. 20388v2 Announce Type: replace-cross Abstract: We derive the asymptotic risk function of regularized empirical risk minimization (ERM) estimators tuned by $n$-fold cross-validation (CV).
arXiv:2607. 09250v1 Announce Type: cross Abstract: The impact of a given training point on a statistical model is classically measured through its leave-one-out influence, which quantifies the effect of its removal from the training set on the model accuracy.
arXiv:2609.23688v1 Announce Type: new Abstract: We study norm-constrained linear classification under Eu clidean adversarial perturbations in a Gaussian model with a low-dimen sional informative subs...
arXiv:2609.39512v1 Announce Type: new Abstract: The small-sample learning problem remains a fundamental challenge in machine learning because limited training data lead to unstable model estimation a...
The paper presents a high‑dimensional theory of learning with data pruning via weighted empirical risk minimization. It analyzes pruning along a one‑dimensional subspace and derives limiting test‑error curves as input dimension and sample size grow with a fixed ratio. The authors show that, in the high‑sample regime, the optimal strategy is to keep only the 2N hardest examples, achieving a Bayes‑optimal error scaling of order N/P instead of the usual sqrt(N/P).