arXiv Machine Learning

The Relative Instability of Model Comparison with Cross-validation

arXiv:2508. 04409v3 Announce Type: replace-cross Abstract: Cross-validation (CV) is known to provide asymptotically exact tests and confidence intervals for model improvement but only when the model comparison is relatively stable.

arXiv AI
6d ago

Robust to Which Model Change? A Unified Evaluation of Robust Counterfactual Explanations

The paper introduces a unified evaluation protocol for robust counterfactual explanations (CFE), testing six robust methods and two baselines across four tabular datasets under eight types of model change. It shows that robustness scores vary by change type and that methods designed for one change family may not transfer to others, with RobX performing most consistently. The study emphasizes the need for a common protocol that defines model changes, measures their impact, and separates generation performance from robustness.

By Marcin Kostrzewa, Maciej Zi\k{e}ba
arXiv Machine Learning
Aug 20

The Road Taken: The Role of Optimizers at the Edge of Stability

The paper investigates the "edge of stability" phenomenon in deep learning, where Hessian eigenvalues remain stable above a classically predicted unstable threshold. It shows that many first‑order optimizers, including gradient descent, can violate this stability bound by up to a factor of 21.1, and that this deviation depends systematically on the optimizer used. The authors propose a new stability threshold based on the directional Hessian and gradient‑alignment score, which removes optimizer‑dependent offsets and offers consistent predictions while providing diagnostic tools to understand how optimizers balance temporal and spatial budgets.

By Jaerin Lee, Kyoung Mu Lee
arXiv Machine Learning
Jul 7

Adversarial LassoNet: Robust Feature Selection via Stability-Driven Sparse Learning

arXiv:2607. 03839v1 Announce Type: new Abstract: Sparse feature selection is critical for high-dimensional machine learning, yet traditional $\ell_1$-regularized methods are often brittle under observational noise and spurious correlations, leading to unstable feature supports and degraded generalization.

By Zhen Huang, Peicheng Xu, Junbiao Pang, Yulong Zheng