arXiv Machine Learning

Models Designed to Forget: Machine Unlearning via Key Deletion

The paper introduces MUNKEY, a memory‑augmented transformer that enables machine unlearning by design. By decoupling instance‑specific memorization from model weights, unlearning is achieved simply by deleting an instance‑identifying key, allowing zero‑shot forgetting without retraining or accessing original data. Experiments on natural image, fine‑grained visual recognition, and medical datasets show that MUNKEY outperforms existing post‑hoc unlearning methods while maintaining predictive performance.

arXiv Machine Learning
Sep 30

Reference-Guided Machine Unlearning

Reference-Guided Machine Unlearning (ReGUn) is a vision unlearning framework that prioritizes distributional indistinguishability over degradation-based heuristics. It uses disjoint held-out data to create a class-conditioned reference distribution for distillation, guiding forget samples toward non-member behavior without explicitly degrading predictions. Experiments across various architectures and datasets show that ReGUn achieves a competitive forgetting–utility trade-off and closely matches retrain-like membership inference behavior.

By Jonas Mirlach, Sonia Laguna, Julia E. Vogt
arXiv Machine Learning
Oct 9

Class Machine Unlearning for Complex Data via Concepts Inference and Data Poisoning

The paper introduces a concept-guided poisoning unlearning framework that explicitly identifies semantic concepts most strongly linked to a target class or knowledge. For image classification, it uses a Post-hoc Concept Bottleneck Model to locate relevant image regions and creates poisoned samples; for large language models, it aggregates Integrated Gradients over multiple question responses to mask important content and generate poisoned training targets. Experiments across several datasets demonstrate effective unlearning while largely preserving model utility.

By Wenhan Chang, Tianqing Zhu, Heng Xu, Wenjian Liu, Wanlei Zhou
arXiv AI
Sep 11

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.

By Ravi Ranjan, Olivera Kotevska, Agoritsa Polyzou
arXiv Machine Learning
Jul 15

Inference-Time Machine Unlearning via Gated Activation Redirection

arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.

By Vin\'icius Conte Turani, Ot\'avio Parraga, Jo\~ao Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinsk\"u
arXiv AI
Aug 26

Restoring Without Forgetting: Continual Learning Across Image Degradations

The paper introduces Restoring without Forgetting (RwF), a continual learning framework for image restoration that handles multiple degradations sequentially without accessing prior data. RwF trains a lightweight adapter for each new degradation, uses an unsupervised routing mechanism to select the correct restoration path, and achieves significant PSNR gains over fine‑tuning on five benchmark degradation domains. The method also demonstrates strong transfer performance on eleven real‑degradation datasets with high routing accuracy.

By Alif Ashrafee, Bartosz Krawczyk
arXiv AI
Oct 26

Concept Acupuncturist: An Efficient and Precise Concept Erasure Method for Diffusion Models via Sparse Autoencoders

Concept Acupuncturist (ConceptACP) is a machine‑unlearning framework that removes targeted concepts from diffusion models efficiently and precisely. It identifies the most relevant cross‑attention block in the U‑Net denoiser, fine‑tunes only that block, and uses Sparse Autoencoders to generate semantically meaningful correction signals for concept erasure. Applied to SDXL‑Turbo, ConceptACP boosts forgetting success by over 36 percentage points while maintaining image quality and prompt fidelity, and works across other Stable Diffusion backbones.

By Yiran Xiahou, Siliang Qin, Kai Chen, Shengzhi Zhang, Renyang Liu