← Back to all news
arXiv Machine Learning

Provable Privacy Attacks on Trained Shallow Neural Networks

The paper investigates provable privacy attacks on trained 2‑layer ReLU neural networks, specifically membership inference and data reconstruction. It demonstrates that the implicit bias of such networks can be leveraged to identify, with high probability, whether a given point was part of the training set in high‑dimensional, nearly orthogonal settings, and to construct a finite set containing a constant fraction of training points in a univariate setting. This work claims to be the first to reveal provable vulnerabilities arising from implicit bias in shallow neural networks.

Related stories

arXiv Computer Vision
6d ago

Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses

The paper "Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses" provides a comprehensive review of model inversion (MI) attacks that exploit trained deep neural networks to reconstruct private training data. It traces the evolution of MI from early machine‑learning contexts to recent DNN‑based attacks across various modalities and learning tasks, offering a detailed taxonomy and comparative analysis of both attacks and defenses. The authors also present an open‑source toolbox on GitHub to support further research in this area.

By Hao Fang, Yixiang Qiu, Hongyao Yu, Wenbo Yu, Jiawei Kong, Baoli Chong, Bin Chen, Xuan Wang, Shu-Tao Xia, Ke Xu
safety
More like this →
arXiv Machine Learning
Aug 28

Provable one-poison backdoor attacks on linear models and ReLU neural networks

The paper demonstrates that a single poisoned data point can successfully create a backdoor in linear models and ReLU neural networks without needing detailed knowledge of the training data. It establishes provable conditions under which this one‑poison attack works with high probability, achieving zero backdooring error while leaving the model’s normal performance largely unaffected. The attack relies only on coarse geometric bounds of the input space and training parameters.

By Thorsten Peinemann, Paula Arnold, Sebastian Berndt, Thomas Eisenbarth, Esfandiar Mohammadi
More like this →
arXiv AI
Sep 18

Batch Normalization Amplifies Memorization and Privacy Risks

Batch Normalization (BN) is widely used to speed up and stabilize deep neural network training, yet its effect on privacy and memorization has been largely unexplored. This study shows that BN significantly increases the memorization of atypical or outlier samples, as evidenced by unintended memorization, per-sample influence, and heightened susceptibility to membership inference attacks across multiple datasets and architectures. A mechanistic analysis of the BN backward pass reveals that BN amplifies the per‑step margin growth of outlier samples during training, thereby intensifying their influence.

By Ngoc Phu Doan, Chongyan Gu, Ihsen Alouani
safety
More like this →
arXiv Machine Learning
4d ago

Empirical Auditing of Edge-Private Graph Generators

The paper presents an empirical audit of privacy leakage in edge‑private graph generators by testing whether outputs from edge‑neighbouring inputs remain distinguishable. It introduces statistically valid lower bounds on privacy loss and compares direct‑edge, local‑structural, and GNN‑based attacks based on the geometry around a target edge. Experiments on two generators and two networks reveal that privacy leakage varies with both the mechanism and the network, and that learned representations expose information beyond conventional local statistics.

By Anum Fatima, Stratis Limnios, James Adams, Lukasz Szpruch, Carsten Maple, Gesine Reinert, Andrew Elliott
safety
More like this →