arXiv Machine Learning
Aug 6
Robustness Emerges Early in Training Dynamics, but Is Not Preserved
arXiv:2608. 04442v1 Announce Type: new Abstract: Robustness to natural corruptions remains a fundamental challenge for deep neural networks.
ALCL: An Adaptive Log-Correntropy Loss for Robust Learning under Non-Gaussian Noise
arXiv:2606. 16050v1 Announce Type: cross Abstract: Robust deep learning under heavy-tailed and impulsive noise remains challenging because conventional losses such as mean squared error (MSE) exhibit unbounded sensitivity to outliers.
Related stories
arXiv AI
Sep 10
DUA-D2C: Dynamic Uncertainty Aware Method for Overfitting Remediation in Deep Learning
The paper introduces DUA-D2C, a Dynamic Uncertainty-Aware Divide2Conquer method that improves overfitting remediation in deep learning. It refines the traditional Divide2Conquer approach by dynamically weighting subset models based on a composite score of accuracy and normalized prediction entropy, allowing the central model to learn more from generalizable and confident edge models. The authors provide theoretical justification, show reduced model variance, and demonstrate significant generalization gains across image, audio, and text benchmarks, even when combined with standard regularizers like Dropout.
arXiv Machine Learning
Jul 8
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
arXiv:2607. 06151v1 Announce Type: new Abstract: Generalization remains a pivotal challenge in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitting and reduced performance on unseen data.
arXiv Machine Learning
Jul 17
Interleaved Noise Injection Improves Clean, Corrupted, and OOD Performance
arXiv:2607. 14466v1 Announce Type: new Abstract: Noise injection is a well-known technique in stochastic optimization.
arXiv AI
Jun 16
Upper Bounds on the Generalization Error of Deep Learning Models via Local Robustness and Stability
arXiv:2606. 16883v1 Announce Type: cross Abstract: Generalization is a critical property of data-driven models, particularly deep learning models deployed in safety-critical applications.
arXiv Machine Learning
Sep 24
CORE-STACK+: Meta-Learning for Deep Stacked Generalization
CORE-STACK+ is a new meta‑learning framework for deep stacked generalization that tackles two key problems in heterogeneous vision ensembles: prediction‑space multicollinearity and calibration collapse. It introduces a four‑step preconditioning pipeline—kernelized redundancy filtering, a lightweight differentiable meta‑feature gate, a spectrum‑adaptive ridge penalty, and a Laplace‑approximate Bayesian blender—to jointly improve conditioning and calibration. Across six vision benchmarks, CORE‑STACK+ boosts accuracy, reduces model count and inference cost, and significantly lowers expected calibration error compared to existing methods.
arXiv Machine Learning
Jun 5
Gradient-Flow Optimization as Dynamic Random-Effects Inference: Testing and Early Stopping with Applications to Deep Learning
arXiv:2605. 27991v2 Announce Type: replace-cross Abstract: Gradient-flow optimization is usually viewed as an algorithmic procedure for minimizing empirical loss, with training duration selected by validation or heuristic early-stopping rules.
arXiv Machine Learning
Aug 24
Generalization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark
arXiv:2602.01718v2 Announce Type: replace Abstract: Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systematic benc...
arXiv AI
Jun 24
Alternate loss functions and regression models that achieve robustness to outliers by modulating the learning rate
arXiv:2606. 22068v2 Announce Type: replace-cross Abstract: Most real-world datasets used for training supervised learning models are contaminated with noisy data and outliers leading to large prediction errors.
arXiv Machine Learning
Jul 9
How Learning Dynamics Drive Adversarially Robust Generalization?
arXiv:2410. 07719v4 Announce Type: replace Abstract: Despite being widely adopted as a canonical framework for learning robust models, adversarial training suffers from robust overfitting.
arXiv AI
Jun 18
Generalized Kullback-Leibler Divergence Loss
arXiv:2503. 08038v2 Announce Type: replace-cross Abstract: In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of (1) a weighted Mean Square Error (wMSE) loss and (2) a Cross-Entropy loss incorporating soft labels.
arXiv Computer Vision
Sep 15
Sparsity-Adaptive Sharpness-Aware Minimization
The paper introduces Sparsity-Adaptive Sharpness-Aware Minimization (SA‑SAM), a method that adjusts the perturbation radius in sharpness-aware training to remain consistent as model sparsity increases. It also evaluates a Magnitude‑Weighted Hessian (MWH) importance metric derived from second‑order analysis. Experiments on CIFAR‑10‑C, CIFAR‑100‑C, and ImageNet‑100‑C show that SA‑SAM improves corruption robustness at 80–90% sparsity while maintaining clean accuracy, and the study reports inference throughput at deployment‑relevant sparsity levels.