Unmerge: Efficient Machine Unlearning via Task Arithmetic
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper introduces Unmerge, an efficient machine unlearning algorithm that treats unlearning as the inverse of task arithmetic. By representing the forget component as a low‑rank basis at each layer, Unmerge optimizes three goals—matching the merged vector, suppressing leakage, and bounding correction size—to limit forget leakage and retain damage. Experiments on ResNet‑50, ViT‑S/16, and Llama‑3.2‑3B show significant performance gains over existing methods while maintaining privacy and feature‑distribution fidelity.
arXiv:2606. 03808v1 Announce Type: cross Abstract: We propose PURGE, a machine unlearning algorithm built on a simple but an under-exploited observation: continual learning (CL) and machine unlearning (MU) which are fundamentally dual problems.
arXiv:2507. 07754v3 Announce Type: replace-cross Abstract: Machine unlearning is usually evaluated by what the classifier outputs: forget-set accuracy, confidence, membership-inference scores.
arXiv:2608. 05783v1 Announce Type: cross Abstract: Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs).
Model Annealing is a data‑efficient learning method that, after an initial memorization phase, spends the remaining training budget on review and consolidation. The training is split into episodes with cyclic learning rates; each episode learns from true labels, tempered predictions stored in a memory of past models, and adversarial ‘dreams’ where the student and memory disagree most. Across several datasets and architectures, Model Annealing improves top‑1 accuracy, likelihood, calibration, and robustness compared to standard cross‑entropy training and matches snapshot distillation while adding no inference overhead.
Machine unlearning aims to remove the influence of specific training data while preserving model utility. Many state-of-the-art approaches pursue this goal by restricting the forgetting update to a subset of parameters selected through gradient-based saliency.