The paper studies distributional drift in nonstationary weighted risk minimization by breaking down excess risk into estimation, drift, and approximation parts. It argues that traditional discrepancy-based bounds are often overly conservative because they can scale larger than the actual excess-risk contribution and react to risk surface changes that do not affect the optimal predictor. The authors provide a tighter oracle drift bound, analyze its behavior in parametrized regression models—particularly linear regression—and extend the results to nonparametric regression.
By Tobias Brock, Thomas Nagler
arXiv:2609. 11918v1 Announce Type: new Abstract: Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples.
By Hongbo Chen, Li Charlie Xia
arXiv:2609. 10767v1 Announce Type: cross Abstract: We develop an exact almost-sure learning theory for smooth parametric models trained by regularly weighted empirical risk minimization on long-range dependent data.
By Elina Moldavskaya
arXiv:2607. 20309v1 Announce Type: cross Abstract: Covariate shift often occurs because, in many real applications, the source and the target observations may be generated from different distributions.
By William Kengne, Ehud Mossa Ockegna
arXiv:2610.07637v1 Announce Type: cross
Abstract: Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture thi...
By Kaito Takanami, Takashi Takahashi, Yoshiyuki Kabashima
arXiv:2607. 11947v1 Announce Type: cross Abstract: Typical semi-supervised learning (SSL) methods rely on distributional assumptions, and their performance degrades when these are violated.
By Yushi Hirose, Hiroo Irobe, Takafumi Kanamori
arXiv:2610.07162v1 Announce Type: new
Abstract: Deep hedging learns trading policies from historical or simulated market trajectories, yet under nonstationarity these training paths may not represent...
By Philipp J. Schneider, Lukas Looser, Antoine Garin, Shuhan Liu, Daniel Kuhn
arXiv:2606. 07382v1 Announce Type: new Abstract: We recast classical shrinkage of high-dimensional covariance estimators as empirical risk minimization over a parametric stochastic interpolant between a source and a target distribution.
By Mathieu Chalvidal, Florentin Coeurdoux, Eric Vanden-Eijnden
The paper proposes an analytic method for determining the optimal early‑stopping time in training neural networks, avoiding the need for gradient‑descent training. It uses Rademacher complexity with an L1‑norm to estimate generalization error, offering a more general approach than previous random‑matrix‑theory based methods. The framework is demonstrated on linear regression and extended to nonlinear neural networks via linear probing, as shown in a MNIST classification example.
By Duy Hoang, Bastien Berret, Olivier Bruneau, Laurent Fribourg
arXiv:2609.39512v1 Announce Type: new
Abstract: The small-sample learning problem remains a fundamental challenge in machine learning because limited training data lead to unstable model estimation a...
By Hong Zheng
The paper introduces a data‑driven Fourier‑trained neural‑network estimator for fixed‑horizon probability densities, using a positive Gaussian–Laplace mixture with a closed‑form characteristic function. Training occurs directly in Fourier space, enabling non‑negativity and unit‑mass preservation while handling both i.i.d. and resampling‑based pseudo‑sampling data. The authors provide theoretical error bounds, a multidimensional extension, and demonstrate competitive empirical performance against Expectation–Maximization, especially on heavy‑tailed targets.
By Duy-Minh Dang, Volter Entoma
Training neural networks requires balancing the trade-off between fitting the training data and achieving robust performance on unseen inputs. This ability, commonly referred to as generalizability, i...