The paper introduces AutoMIA, a framework that uses large language model agents to automatically design and implement new membership inference attack (MIA) signal computations. By systematically exploring a wide range of attack strategies, AutoMIA discovers novel MIAs tailored to specific target models and datasets, achieving up to a 0.18 absolute improvement in AUC over existing methods. This demonstrates that LLM agents can serve as an effective and scalable approach for creating state‑of‑the‑art MIAs.
arXiv:2606. 17464v1 Announce Type: new Abstract: Membership inference attacks (MIAs) are a canonical way to assess a machine learning model's privacy properties.
The paper examines the reliability of membership inference attack (MIA) vulnerability evaluation. It identifies two weaknesses: finite‑sample bias from sampling shadow datasets from a fixed superset, and miscalibration when aggregating true positive rates across individuals at very low false positive rates. The authors propose simple fixes that avoid extra computational cost and suggest further improvements with additional computation.
arXiv:2606. 14210v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in privacy-sensitive domains, where users must balance the risk of data exposure through external APIs against the high computational cost of local deployment.
arXiv:2606. 17110v1 Announce Type: cross Abstract: Large Language Models are increasingly trained on proprietary or sensitive data, from private healthcare and financial records to user conversations containing secrets.
UniGuardian is a training‑free detector for large language models that jointly identifies prompt injection, backdoor, and adversarial attacks—collectively called Prompt Trigger Attacks (PTA). It measures how structured prompt perturbations shift the model’s output distribution and uses a single‑forward strategy to detect attacks while generating text in a shared batched forward pass. Experiments show that UniGuardian accurately and efficiently identifies trigger‑activated prompts in LLMs.
arXiv:2606. 31991v1 Announce Type: cross Abstract: The tendency of large generative models to memorize training data makes sample verification critical for privacy auditing and copyright enforcement.
arXiv:2605. 26595v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison.
The paper introduces a black-box membership inference attack framework tailored for fine-tuned text-to-speech models, addressing challenges in query generation and representation engineering. It evaluates five query types, finding recitation queries most effective, and uses multi-level speech embeddings with temporal alignment for fine-grained comparison. Experiments on CosyVoice2, F5-TTS, and XTTS-v2 trained on VCTK and British Dialect datasets show high privacy leakage, with speaker-level AUC above 0.80 and record-level AUC between 0.80 and 0.90.
arXiv:2606. 15788v1 Announce Type: cross Abstract: Large Language Models (LLMs) constitute pivotal components within the AI-dominated information technology ecosystem.
arXiv:2608. 10171v1 Announce Type: new Abstract: The rapid advancement of Large Language Models (LLMs) has facilitated their ubiquitous integration into various domains, leading to widespread adoption.
arXiv:2608. 00732v1 Announce Type: new Abstract: Backdoor attacks pose a serious threat to deep neural networks, especially when training relies on third-party data, allowing adversaries to inject malicious behaviors through data poisoning.
arXiv:2506. 06488v3 Announce Type: replace Abstract: A key tool in developing safe AI models is \emph{data auditing}, i.