arXiv AI October 19, 2026 By Mohamed Shaaban, Mohamed Elmahallawy

Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning

The paper introduces JEPA‑Prune, a post‑hoc neuron‑pruning method that mitigates private data leakage in fine‑tuned large language models. It trains a public model on PII‑free surrogate views using a joint‑embedding predictive architecture, then uses cross‑model representation probes, integrated gradients, and a specificity score to identify and prune neurons that respond to personally identifiable information. Experiments show that pruning only 1.6% of neurons can dramatically reduce PII inference accuracy and recall while preserving model utility, outperforming existing privacy‑neuron editing techniques.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 17

CutClean: Neural Network Pruning for Privacy-Preserving Inference

arXiv:2608. 13773v1 Announce Type: cross Abstract: Neural networks are increasingly deployed in high-stakes applications with growing privacy leakage concerns.

By Leonardo Magliolo, Vito Paolo Pastore, Giuseppe Valenzise, Enzo Tartaglione

efficiency safety

arXiv AI

Aug 25

Reinforcement Learning on Benign Facts Amplifies Leakage of Memorized Private Data

arXiv:2608.21727v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is deployed to make models better at reasoning tasks, but its side effect on what models will d...

By Renfei Zhang, Niloofar Mireshghallah

reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 16

When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training

arXiv:2607. 13541v1 Announce Type: cross Abstract: To overcome data scarcity and privacy constraints in data collection, it has become standard practice across academia and industry to augment real training data with text-to-image (T2I)-generated synthetic data, a paradigm we term Real-Synthetic Mix-Training (RSMT).

By Na Li, Boyu Kuang, Hongsheng Hu, Liquan Chen, Hyoungshick Kim, Yansong Gao, Anmin Fu

diffusion safety

arXiv AI

Sep 25

Decoupling Knowledge and Privacy: Post-Task Self-Distillation Replay for LLM Continual Learning

The paper introduces SPARK, a method for privacy‑preserving continual learning that decouples knowledge retention from privacy correction. SPARK freezes the post‑task distribution and then selectively corrects it to reduce the likelihood of sensitive content while maintaining strong performance on current and past tasks. Experiments show that this approach effectively suppresses PII and preserves continual‑learning utility across various settings.

By Shengtao Wen, Yunying Yang, Xiang Chen, Lingbing Guo, Yu Tian, Sheng-Jun Huang

llms efficiency safety data-engineering

arXiv Machine Learning

Jun 5

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

arXiv:2605. 07482v2 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining.

By Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei, Mohammad Rostami, Jesse Thomason, Robin Jia

llms efficiency benchmarks

arXiv Machine Learning

Aug 27

Reconstruction of Personally Identifiable Information from Proprietary Data in Supervised Fine-Tuned Models

The paper examines how supervised fine-tuning (SFT) of large language models can leak personally identifiable information (PII) when the fine-tuning data contains user-provided sensitive details. It introduces COVA, a coverage-aware decoding algorithm that improves targeted PII reconstruction from SFT models, especially when an adversary has limited contextual knowledge about a target. Experiments on medical and legal Q&A datasets show that even small proprietary SFT datasets can lead to significant privacy leakage via PII reconstruction.

By Sae Furukawa, Alina Oprea

llms fine-tuning safety

One email a morning, machine-written

Related stories

CutClean: Neural Network Pruning for Privacy-Preserving Inference

Reinforcement Learning on Benign Facts Amplifies Leakage of Memorized Private Data

When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training

Decoupling Knowledge and Privacy: Post-Task Self-Distillation Replay for LLM Continual Learning

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

Reconstruction of Personally Identifiable Information from Proprietary Data in Supervised Fine-Tuned Models