The paper analyzes Gradient Descent (GD) on a minimal binary classification problem with a two‑neuron ReLU network trained on two instances. It proves that GD converges to an optimal robustness margin—maximizing the distance between the decision boundary and training points—but does so at a very slow rate, specifically θ(1/ln(t)). Empirical simulations confirm this slow convergence across various initializations, and the authors provide a rigorous analysis of GD trajectories across different activation patterns to establish the first explicit lower bound on robustness convergence in a non‑linear model.
arXiv:2606. 16883v1 Announce Type: cross Abstract: Generalization is a critical property of data-driven models, particularly deep learning models deployed in safety-critical applications.
arXiv:2610. 23007v1 Announce Type: new Abstract: Gradient descent (GD) with large learning rates can successfully optimize neural networks even though classical smoothness-based learning rate conditions are violated.
arXiv:2410. 07719v4 Announce Type: replace Abstract: Despite being widely adopted as a canonical framework for learning robust models, adversarial training suffers from robust overfitting.
arXiv:2606. 04476v1 Announce Type: new Abstract: In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.
arXiv:2605. 29497v2 Announce Type: replace Abstract: We study the problem of robustly learning Gaussian Single Index Models (SIMs) in the presence of heavy-tailed noise and a constant fraction of adversarially corrupted covariates and responses.
arXiv:2610.28561v1 Announce Type: new Abstract: Stochastic gradient descent (SGD) is the cornerstone of modern deep learning, yet its theoretical convergence properties remain poorly understood in no...
arXiv:2301. 06308v2 Announce Type: replace-cross Abstract: Sharpness-aware minimization (SAM) is a training method that seeks to find flat minima in deep learning, resulting in state-of-the-art performance across various domains.
arXiv:2606. 31653v1 Announce Type: cross Abstract: Certified training aims to produce models whose predictions can be formally verified against adversarial perturbations, typically by optimising upper bounds on the worst-case loss over an allowed perturbation set.
arXiv:2608. 04442v1 Announce Type: new Abstract: Robustness to natural corruptions remains a fundamental challenge for deep neural networks.
arXiv:2608. 04173v1 Announce Type: new Abstract: Deep neural networks (DNNs) deployed on resource-constrained neuromorphic hardware face three concurrent challenges: the need for model compression through pruning, vulnerability to adversarial input perturbations, and susceptibility to hardware-induced weight faults such as stuck-at-zero errors.
arXiv:2610.20439v1 Announce Type: new Abstract: We study classification dynamics under small initialization and identify a pronounced non-monotone behavior of accuracy that we call the accuracy well:...
arXiv:2608. 14594v1 Announce Type: new Abstract: Projected Gradient Descent (PGD) is widely used to evaluate adversarial robustness, typically via final adversarial accuracy, which does not capture model behaviour throughout the attack.