arXiv Machine Learning

Extracting Forgotten Prompts from Targeted Unlearned Models

The paper introduces Targeted Active Search (TAS), a black-box attack that extracts forgotten prompts from models that have undergone unlearning. TAS first identifies forgotten entities by querying the model with canonical templates and an entity pool, then reconstructs the prompts by instantiating templates with these entities. Experiments on three unlearning methods, datasets, and large language models demonstrate that TAS can recover 100% of forgotten entities and up to 95% of prompts while using far fewer queries than naive probing.

arXiv Computation and Language
Aug 25

Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation

arXiv:2608.21606v1 Announce Type: new Abstract: Machine unlearning aims to remove the influence of targeted training data from a model while preserving its remaining capabilities, but evaluating whet...

By Ayush Gupta, Hima Varshini Surisetty, Sreevidya Bollineni, Varad Ingale, Tuhina Tripathi, Abhishek Lalwani, Somya Chatterjee, Sadid Hasan
arXiv AI
Sep 16

RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distribution

RAG-CT is a defense mechanism designed to protect Retrieval-Augmented Generation (RAG) systems from leaking personally identifiable information (PII). It works by detecting malicious queries through analysis of entropy and margin distributions, applying a score-based detection method. Experiments across four attack strategies and four baseline defenses on two datasets show that RAG-CT significantly reduces PII leakage while outperforming existing defenses, all without altering the underlying LLM or retriever.

By Xingyu Lyu, Jiayimei Wang, Jianfeng He, Ning Wang, Yidan Hu, Yimin Chen
arXiv AI
Jun 9

Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions

arXiv:2604. 08304v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) extends large language models (LLMs) with external knowledge, but this access path also introduces security risks that existing work often conflates with inherent LLM flaws.

By Yuming Xu, Mingtao Zhang, Zhuohan Ge, Haoyang Li, Nicole Hu, Yongqi Zhang, Zhiyuan Wen, Jason Chen Zhang, Qing Li, Lei Chen
arXiv Machine Learning
Jul 3

Embedding Inference Attack

arXiv:2607. 01276v1 Announce Type: cross Abstract: Embedding models are essential components of modern Information Retrieval (IR) systems, yet they are typically hidden behind APIs.

By Cedric Fitiavana Raelijohn, S\'ebastien Gambs, Jean-Francois Rajotte