arXiv Machine Learning

SoK: Colluding Adversaries in Machine Learning Pipelines

arXiv:2606. 10091v1 Announce Type: cross Abstract: Machine learning (ML) models are susceptible to various security, privacy, and fairness risks.

arXiv Computer Vision
Sep 21

Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses

The paper "Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses" provides a comprehensive review of model inversion (MI) attacks that exploit trained deep neural networks to reconstruct private training data. It traces the evolution of MI from early machine‑learning contexts to recent DNN‑based attacks across various modalities and learning tasks, offering a detailed taxonomy and comparative analysis of both attacks and defenses. The authors also present an open‑source toolbox on GitHub to support further research in this area.

By Hao Fang, Yixiang Qiu, Hongyao Yu, Wenbo Yu, Jiawei Kong, Baoli Chong, Bin Chen, Xuan Wang, Shu-Tao Xia, Ke Xu
arXiv Machine Learning
Oct 23

Does More Compute Make Every Adversarial Attack Transferable?

The paper investigates whether increased computational resources for an attacker can make adversarial attacks transferable across all defenders. It finds that for classification tasks, a quadratic resource gap guarantees either a transferable attack or a defense, whereas for generative tasks this guarantee fails. The authors also show that a defense that adds computation after seeing the input can be provably stronger than any fixed-model defense, suggesting that inference-time compute can enhance robustness.

By Greg Gluch, Shafi Goldwasser, Anastasiia Struss
arXiv AI
Jul 21

A Survey on Unlearnable Data

arXiv:2503. 23536v3 Announce Type: replace-cross Abstract: Unlearnable data (ULD) has emerged as an innovative defense technique to prevent machine learning models from learning meaningful patterns from specific data, thus protecting data privacy and security.

By Jiahao Li, Yiqiang Chen, Yunbing Xing, Yang Gu, Xiangyuan Lan
arXiv AI
Sep 1

Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning

The paper introduces FAB, an attack that uses meta‑learning to embed dormant adversarial behaviors into large language models (LLMs). These behaviors remain inactive until the model is finetuned by downstream users, at which point the model can exhibit unwanted actions such as unsolicited advertising, jailbreakability, or over‑refusal. FAB is shown to be effective across multiple LLMs and resilient to various finetuning settings.

By Thibaud Gloaguen, Mark Vero, Robin Staab, Martin Vechev