AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Statistics ML
Oct 1

A Unified Dual Method for Matching Problems

arXiv:2609.39339v1 Announce Type: cross Abstract: Matching problems are ubiquitous in data science as they enable the alignment of structured objects and distributions. While existing solvers are oft...

By Guillaume Houry (HeKA | U1346), Ferdinand Genans (SU, LPSM), Jean Feydy (HeKA | U1346), Fran\c{c}ois-Xavier Vialard (LIGM)
arXiv Statistics ML
Oct 1

Grokking through the Lens of Minimum-Norm Interpolation

The paper develops a statistical theory for minimum‑norm interpolation in high‑dimensional regression, showing how regularization geometry and signal sparsity affect generalization. It identifies regimes where sparsity‑promoting regularizers yield exact interpolation that is far more accurate than approximate fitting, and proves a zero–one generalization law for strongly overparameterized noiseless problems. The authors also characterize training and generalization errors along ρ‑regularization paths when feature dimension and sample size are proportional, demonstrating that generalization improves with more sparsity‑promoting norms and sparser targets, and that small changes in regularization strength can cause large shifts in generalization. whyItMatters":"The work provides a quantitative understanding of delayed generalization (grokking) and reveals a statistical instability in minimum‑norm interpolation, offering insights that could guide the design of regularizers for better generalization in overparameterized models."

By Gil Kur, Ileana Rugina, Cl\'ementine Carla Juliette Domin\'e, Marco Mondelli
arXiv Computer Vision
Oct 1

Anchoring Adversarial Trajectories to Data Manifolds: A Bilevel Transfer Optimization Framework

The paper introduces Manifold Anchored Bilevel Transfer (MABT), a framework that aligns adversarial attack trajectories with the intrinsic data manifold to reduce surrogate-specific overfitting. MABT employs a relaxed manifold-anchoring operator as a semantic rectifier and formulates transfer attack generation as a distributional bilevel optimization problem, learning geometry-aligned initializations that minimize expected transfer risk. A Hessian-free solver with linear-time complexity is developed to efficiently solve the resulting hierarchy, and experiments show improved transferability across 10 attackers, 28 configurations, various victim models, and defense mechanisms.

By Yaohua Liu, Yifan Guo, Jiaxin Gao
arXiv AI
Oct 1

Probabilistic Adversarial Training

The paper introduces probabilistic adversarial training, a method that enhances robustness by reducing the overlap between a distance-based distribution and a victim-classifier-induced distribution. It derives a KL-based lower bound on probabilistic robustness, which serves as a tractable surrogate objective. Experiments confirm that this approach consistently improves probabilistic robustness and can even boost non-probabilistic adversarial training methods through an induced scaling factor.

By Andi Zhang, Xingyu Zhao, Siddartha Khastgir