← Back to all news
arXiv Statistics ML

Beyond Discrepancy: Oracle Drift in Nonstationary Weighted Risk Minimization

The paper studies distributional drift in nonstationary weighted risk minimization by breaking down excess risk into estimation, drift, and approximation parts. It argues that traditional discrepancy-based bounds are often overly conservative because they can scale larger than the actual excess-risk contribution and react to risk surface changes that do not affect the optimal predictor. The authors provide a tighter oracle drift bound, analyze its behavior in parametrized regression models—particularly linear regression—and extend the results to nonparametric regression.

Related stories

arXiv Machine Learning
Oct 12

Fast Rates for Nonstationary Weighted Risk Minimization

The paper investigates weighted empirical risk minimization under nonstationary data, decomposing the excess risk into a learning component and a drift component. It establishes oracle inequalities for the learning error under mixing conditions, uniformly over arbitrary weight classes, and derives explicit bounds for drift errors in square loss settings, covering structural‑break and smoothly drifting processes. The analysis is extended to auto‑regression problems with linear models, basis approximations, and neural networks, revealing that optimal weights balance learning and drift via a bias–variance trade‑off and recover minimax‑optimal rates in unweighted, stationary cases.

By Tobias Brock, Thomas Nagler
safety
More like this →
arXiv Machine Learning
Sep 24

CVaR anchor regression protects against rare shifts

The paper introduces CVaR anchor regression, a method that replaces the average squared mean residual used in anchor regression with a tail average to better protect against rare, large shifts in training data. This approach avoids over‑weighting environments with high noise levels and, under a linear structural model with heteroscedastic noise, provides an exact worst‑case risk guarantee. For discrete environments, decreasing the CVaR tail fraction expands the robustness set from an ellipsoid to a scaled convex hull of training shifts and their negatives, allowing improved protection against rare shifts while maintaining accuracy on common environments, as demonstrated on New York City taxi data.

By Malte Londschien
More like this →
arXiv Machine Learning
Sep 11

General Quantification of Covariate and Concept Shifts

arXiv:2609. 11918v1 Announce Type: new Abstract: Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples.

By Hongbo Chen, Li Charlie Xia
More like this →
arXiv Machine Learning
Jul 27

Smart predict-then-robustly-optimize

arXiv:2607. 21773v1 Announce Type: new Abstract: In this paper, we propose and study a robust variant of the smart predict-then-optimize approach that accounts for prediction shifts due to disturbance in the covariate feature space.

By Aakil Caunhye, Xuefei Lu, Belen Martin-Barragan
More like this →