arXiv Machine Learning

Tail-Weight Control and Localized Generalization in Nearly Low-Rank Adversarial Classification

arXiv Machine Learning
Sep 22

Statistical Inference for Adversarial Training: Central Limit Theorems via Optimal Transport

The paper rigorously analyzes the statistical and learning-theoretic properties of adversarial training models for classification, focusing on empirical optimal partial transport. It establishes two central limit theorems—one centered at the expected empirical value and another at the population value with smoothing—by leveraging the uniqueness of optimal potentials across various optimal transport formulations and empirical process theory. In the binary setting, the authors prove uniqueness of the optimal potential via a connection to multi-marginal optimal transport, and as additional results they derive stability of the saddle point, sample complexity, and concentration bounds for generalization error.

By Kim Jakwang, Kwon Dohyun
Hugging Face Trending Papers
Aug 10

Confusion-Geometry Rebalancing for Long-Tailed Adversarial Training

Adversarial training under long tailed distributions suffers from a dual imbalance: the class imbalance skews the training objective toward head classes, and the adversarial inner maximization may further amplify this bias. Existing methods mitigate this issue by correcting class priors or adapting class wise robust supervision, yet they treat each class in isolation and fail to identify which boundaries drive long tailed collapse.

arXiv Machine Learning
Nov 6

Adversarial Training with Large Step Sizes: Implicit Bias, Optimization, and Sharpness

The paper investigates how deterministic Gradient Descent behaves when applied to Adversarial Training with large, constant step sizes. It analyzes both a linear logistic regression setting—examining where parameters converge (implicit bias) and how fast losses decrease (optimization)—and neural network training, focusing on the evolution of loss sharpness. The study reveals three key distinctions from Standard Training: the implicit bias gap persists even with large steps, there is no uniform convergence rate for optimization error, and sharpness follows a two‑stage dynamic unlike the Edge‑of‑Stability pattern seen in Standard Training.

By Yi Feng, Andrea Paudice
arXiv AI
Sep 18

Information-Geometric Inverse Distillation for Enhancing Adversarial Transferability

The paper introduces Inverse Knowledge Distillation (IKD), an attack‑agnostic technique that enhances adversarial transferability by maximizing the discrepancy between benign and adversarial prediction distributions on a surrogate model. IKD employs a CE/KL‑equivalent soft‑label objective to push adversarial predictions away from a fixed benign anchor, leveraging Fisher‑sensitive surrogate directions. The authors provide theoretical analysis showing CE and KL induce identical gradients, derive a lower bound on Fisher‑subspace overlap, and demonstrate through extensive ImageNet experiments that IKD consistently improves black‑box attack performance across CNN, ViT, and defended models.

By Wenyuan Wu, Yuan Sun, Yingke Chen, Chao Su, Xi Peng, Dezhong Peng, Xu Wang
arXiv Machine Learning
6d ago

Structured Confidence Perturbation via Fisher Geometry for Adversarial Training

The paper introduces two methods to improve adversarial training by targeting the model’s confidence structure. One method blends cross‑entropy with KL divergence to simultaneously degrade accuracy and distribution consistency, while the other uses Fisher geometry to generate perturbations that maximize distribution distance on the confidence manifold. Experiments show these techniques outperform existing approaches in robustness and transferability, especially for high‑confidence structural perturbations.

By Chenlong Wei, Hanling Wang, Chong Zhang, Qinkai Yu, Fangyu Wu, Xiaobo Jin