Conformal Privacy Auditing (CPA) is a distribution‑free framework that calibrates re‑identification risk for each released document against large language model (LLM)‑empowered adversaries. It outputs a conformal ambiguity set of candidate identities that is guaranteed to contain the true identity with a user‑chosen confidence level under exchangeability, along with an interpretable leakage proxy derived from the set size. CPA supports both logit‑access and sampling‑only attackers, enabling audits of both open‑source and proprietary models, and demonstrates calibrated coverage across various benchmarks and attacker configurations.
arXiv:2606. 09401v1 Announce Type: new Abstract: Recent work has applied differential privacy (DP) to adapt large language models (LLMs) for sensitive applications, offering theoretical guarantees.
arXiv:2606. 16952v2 Announce Type: replace-cross Abstract: The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets.
The paper introduces LLM‑Attacker, a unified framework for evaluating privacy risks in split and distributed large language models. It records attacker choices—such as message types, model weights, datasets, and recovery metrics—to enable consistent comparison and extension of privacy evaluations. Experiments on GPT‑2 show that providing training gradients boosts token recovery from 94.20% to 97.38% and exact‑document recovery from 13.71% to 37.77%, illustrating the framework’s ability to quantify how changes in attacker access affect privacy outcomes.
The paper introduces LLM‑Attack Suite, a unified framework for evaluating privacy risks in split and distributed large language models. It records attacker choices—such as message types, model weights, datasets, and recovery metrics—to enable consistent comparison and extension of privacy evaluations. Experiments on GPT‑2 demonstrate that providing gradients during training significantly boosts token and document recovery rates, illustrating how attacker access levels influence privacy outcomes.
arXiv:2606. 10481v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning of large language models (LLMs) can exhibit problematic memorization of individual training examples.
arXiv:2509. 20324v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) is an emerging approach in natural language processing that combines large language models (LLMs) with external document retrieval to produce more accurate and grounded responses.
arXiv:2606. 16952v1 Announce Type: cross Abstract: The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets.
arXiv:2608. 09001v1 Announce Type: cross Abstract: Black-box privacy scores for retrieval-augmented generation (RAG) are difficult to interpret unless the audited defense's active pipeline hook is known.
arXiv:2606. 24408v1 Announce Type: new Abstract: Assessing the privacy of large language models (LLMs) presents significant challenges.
The paper surveys attacks and defenses in Retrieval-Augmented Generation (RAG), a technique that improves large language models by grounding outputs in external knowledge. It identifies new robustness and security risks such as corpus poisoning, backdoor attacks, privacy leakage, and fairness violations, and notes that existing surveys inadequately cover attacker objectives, threat models, and stage-specific defenses. The survey offers a unified, pipeline-aware overview, formalizing threat models across the corpus, retriever, and generator, categorizing attacks by accuracy, privacy, and fairness, and reviewing defenses for retrieval, rerank, generation, and traceback stages, while also summarizing robustness benchmarks and explainability methods.
arXiv:2606. 28962v1 Announce Type: cross Abstract: Model quantization is essential for the efficient deployment of Large Language Models (LLMs), but introduces a critical vulnerability: Quantization-Conditioned Backdoor (QCB) attacks.
The paper examines how supervised fine-tuning (SFT) of large language models can leak personally identifiable information (PII) when the fine-tuning data contains user-provided sensitive details. It introduces COVA, a coverage-aware decoding algorithm that improves targeted PII reconstruction from SFT models, especially when an adversary has limited contextual knowledge about a target. Experiments on medical and legal Q&A datasets show that even small proprietary SFT datasets can lead to significant privacy leakage via PII reconstruction.