arXiv Machine Learning
Catastrophic Overfitting is Weight Drift the Training Loss Cannot See
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
2d ago
Catastrophic Overfitting is Weight Drift the Training Loss Cannot See
Fast adversarial training (FAT) makes robust training affordable by replacing the multi-step attack with a single-step one, but it suffers from catastrophic overfitting (CO): partway through training,...
arXiv AI
Jun 2
SORA: Free Second-Order Attacks in Fast Adversarial Training
arXiv:2606. 00738v1 Announce Type: cross Abstract: Adversarial Training (AT) is a leading defense against adversarial examples but often suffers from Catastrophic Overfitting (CO) in efficient single-step variants, where robustness to multi-step attacks collapses despite high single-step performance.
arXiv AI
Oct 26
Catastrophic Overfitting, Entropy Gap and Participation Ratio: A Noiseless $l^p$ Norm Solution for Fast Adversarial Training
arXiv:2505. 02360v3 Announce Type: replace-cross Abstract: Adversarial training is a cornerstone of robust deep learning, but fast methods like the Fast Gradient Sign Method (FGSM) often suffer from Catastrophic Overfitting (CO), where models become robust to single-step attacks but fail against multi-step variants.
arXiv AI
Jul 28
Scale Weight Decay and Train Better
arXiv:2607. 23777v1 Announce Type: cross Abstract: The discovery of scaling laws has motivated training neural networks on ever increasing quantities of data.
arXiv Machine Learning
Sep 24
Median Temporal Ensembling: Training-Free Robust Aggregation for Action-Chunked Visuomotor Policies
The paper introduces Median Temporal Ensembling, a training‑free aggregation method for action‑chunked visuomotor policies that replaces the standard exponential weighted mean with a coordinate‑wise median. This approach remains robust against adversarial corruption, maintaining a high recovery rate even as attack strength increases, and performs at least as well as the mean across numerous configurations while improving in many cases. It also handles non‑adversarial failures such as blank camera frames and shows limited impact on clean data, though it cannot counteract uniform shifts applied to all predictions.
arXiv Machine Learning
Aug 7
Threshold-Based Early Stopping of Accumulations in Neural Networks with Binary Activation
arXiv:2608. 06177v1 Announce Type: new Abstract: Binary neural networks are very attractive for constrained deployment, enabling small footprint and low-power inference.