arXiv:2508.20443v3 Announce Type: replace Abstract: Large language models (LLMs) are trained on massive datasets that may include private or copyrighted content. Due to growing privacy and ownership...
arXiv:2507. 07754v3 Announce Type: replace-cross Abstract: Machine unlearning is usually evaluated by what the classifier outputs: forget-set accuracy, confidence, membership-inference scores.
Reference-Guided Machine Unlearning (ReGUn) is a vision unlearning framework that prioritizes distributional indistinguishability over degradation-based heuristics. It uses disjoint held-out data to create a class-conditioned reference distribution for distillation, guiding forget samples toward non-member behavior without explicitly degrading predictions. Experiments across various architectures and datasets show that ReGUn achieves a competitive forgetting–utility trade-off and closely matches retrain-like membership inference behavior.
The paper investigates whether representational entanglement—shared structure between knowledge domains—impedes unlearning in neural networks. Using Selective Gradient Masking, the authors train six 254M‑parameter language models with varying degrees of disentanglement between biology and non‑biology knowledge, then apply three standard unlearning methods to each. Results show that more disentangled models consistently achieve better retain‑forget trade‑offs, with up to four‑fold lower retain cost at the same forgetting level, providing direct evidence that entanglement contributes to collateral damage in unlearning.
The paper investigates whether representational entanglement—shared structure between knowledge domains—impedes unlearning in neural networks. By training six 254M‑parameter language models with varying degrees of disentanglement between biology and non‑biology knowledge and applying three unlearning methods, the authors find that more disentangled models consistently achieve better retain‑forget trade‑offs, with up to four‑fold lower retain cost. This controlled experiment provides direct evidence that entanglement contributes to collateral damage during unlearning, supporting a long‑standing hypothesis in interpretability research.
arXiv:2609.05966v1 Announce Type: new Abstract: Existing unlearning approaches typically rely on post hoc weight adaptation or distillation, leading to duplicated memory costs, degraded generalizatio...
The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.
arXiv:2606. 20431v1 Announce Type: new Abstract: Continual learning (CL) systems often forget previously acquired knowledge, yet the mechanisms driving forgetting remain hard to isolate in practice because real datasets entangle many factors.
arXiv:2609.36612v1 Announce Type: new Abstract: Unlearning in large language models (LLMs) is typically evaluated at the output level, where a model appears to suppress sensitive or undesirable conte...
arXiv:2511. 20196v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) can inadvertently memorize privacy-sensitive information during training.
UNBIND is a code unlearning framework that selectively removes memorized code from large language models at inference time while keeping the model weights unchanged. It constructs separate directional steering for hidden states that correspond to target code, enabling high forgetting rates (97.3–99.1% reduction in target code reproduction) with minimal loss in programming utility. Across multiple baselines, corpora, and evaluation metrics—including F‑BLEU, HumanEval+, and MBPP+—UNBIND consistently achieves the best joint forgetting and utility scores, and it effectively eliminates long exact code spans in repeated extraction tests.
Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers...