arXiv Machine Learning August 27, 2026 By Sae Furukawa, Alina Oprea

Reconstruction of Personally Identifiable Information from Proprietary Data in Supervised Fine-Tuned Models

Read the original on arXiv Machine Learning →

The paper examines how supervised fine-tuning (SFT) of large language models can leak personally identifiable information (PII) when the fine-tuning data contains user-provided sensitive details. It introduces COVA, a coverage-aware decoding algorithm that improves targeted PII reconstruction from SFT models, especially when an adversary has limited contextual knowledge about a target. Experiments on medical and legal Q&A datasets show that even small proprietary SFT datasets can lead to significant privacy leakage via PII reconstruction.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 28

PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window

arXiv:2607. 22695v1 Announce Type: new Abstract: Large Language Models (LLMs) are capable of generalizing human language for the completion of never-before-seen tasks, leading to widespread deployment.

By Ryan Thornton, Mir Mehedi Ahsan Pritom, Maanak Gupta

llms benchmarks safety

arXiv Machine Learning

Jul 21

Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering

arXiv:2310. 16152v5 Announce Type: replace-cross Abstract: Federated learning (FL) has become a key component in various language modeling applications such as machine translation, next-word prediction, and medical record analysis.

By Md Rafi Ur Rashid, Vishnu Asutosh Dasu, Kang Gu, Najrin Sultana, Shagufta Mehnaz

llms nlp safety

arXiv Machine Learning

Jun 17

Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs

arXiv:2606. 17110v1 Announce Type: cross Abstract: Large Language Models are increasingly trained on proprietary or sensitive data, from private healthcare and financial records to user conversations containing secrets.

By Md Abdullah Al Mamun, Ngoc Phu Doan, Pedram Zaree, Ihsen Alouani, Nael Abu-Ghazaleh

llms multimodal safety

arXiv AI

Sep 16

RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distribution

RAG-CT is a defense mechanism designed to protect Retrieval-Augmented Generation (RAG) systems from leaking personally identifiable information (PII). It works by detecting malicious queries through analysis of entropy and margin distributions, applying a score-based detection method. Experiments across four attack strategies and four baseline defenses on two datasets show that RAG-CT significantly reduces PII leakage while outperforming existing defenses, all without altering the underlying LLM or retriever.

By Xingyu Lyu, Jiayimei Wang, Jianfeng He, Ning Wang, Yidan Hu, Yimin Chen

llms rag benchmarks safety

arXiv Machine Learning

Aug 21

Inadvertent Context Leakage in Language Models

arXiv:2608. 19857v1 Announce Type: new Abstract: For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data.

By Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri, Sanjam Garg, Saeed Mahloujifar

llms agents reinforcement-learning safety

arXiv AI

Jul 22

Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data

arXiv:2606. 16952v2 Announce Type: replace-cross Abstract: The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets.

By Kareem Amin, Rudrajit Das, Alessandro Epasto, Adel Javanmard, Dennis Kraft, M\'onica Ribero, Sergei Vassilvitskii

llms safety

One email a morning, machine-written

Related stories

PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window

Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering

Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs

RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distribution

Inadvertent Context Leakage in Language Models

Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data