← Back to all news
arXiv Machine Learning By Georgios Politis, Evangelos Pappas

The LLM-Attacker: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems

Read the original on arXiv Machine Learning →

The paper introduces LLM‑Attacker, a unified framework for evaluating privacy risks in split and distributed large language models. It records attacker choices—such as message types, model weights, datasets, and recovery metrics—to enable consistent comparison and extension of privacy evaluations. Experiments on GPT‑2 show that providing training gradients boosts token recovery from 94.20% to 97.38% and exact‑document recovery from 13.71% to 37.77%, illustrating the framework’s ability to quantify how changes in attacker access affect privacy outcomes.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 21

LLM-Attack Suite: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems

The paper introduces LLM‑Attack Suite, a unified framework for evaluating privacy risks in split and distributed large language models. It records attacker choices—such as message types, model weights, datasets, and recovery metrics—to enable consistent comparison and extension of privacy evaluations. Experiments on GPT‑2 demonstrate that providing gradients during training significantly boosts token and document recovery rates, illustrating how attacker access levels influence privacy outcomes.

By Georgios Politis, Evangelos Pappas
llmssafety
More like this →
arXiv Computation and Language
Sep 21

Conformal Privacy Auditing: Calibrated Re-identification Attacks with Statistical Guarantees

Conformal Privacy Auditing (CPA) is a distribution‑free framework that calibrates re‑identification risk for each released document against large language model (LLM)‑empowered adversaries. It outputs a conformal ambiguity set of candidate identities that is guaranteed to contain the true identity with a user‑chosen confidence level under exchangeability, along with an interpretable leakage proxy derived from the set size. CPA supports both logit‑access and sampling‑only attackers, enabling audits of both open‑source and proprietary models, and demonstrates calibrated coverage across various benchmarks and attacker configurations.

By Shuo Huang, Gholamreza Haffari, Xingliang Yuan, Ting Yu, Lizhen Qu
llmsbenchmarkssafety
More like this →