Large language model unlearning aims to suppress designated undesirable knowledge while preserving benign capabilities. Many unlearning objectives focus on suppressing undesired answers, while recent target-guided variants specify replacement behavior but still leave update locality largely unconstrained.
arXiv:2606. 10989v1 Announce Type: new Abstract: Large language model unlearning aims to suppress designated undesirable knowledge while preserving benign capabilities.
The paper introduces FDCU, a dual‑constrained subspace projection framework designed to improve machine unlearning for large language models. FDCU limits parameter updates with a dual‑masking rule that preserves general knowledge via Fisher Information while preventing the activation of spurious suppressors through the Principle of Minimal Functional Intervention. Experiments show that FDCU achieves state‑of‑the‑art robustness against retraining attacks while maintaining near‑lossless general utility, thereby ensuring durable safety for LLMs.
arXiv:2606. 06320v1 Announce Type: new Abstract: Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities.
The paper introduces EoupCT, a framework that estimates and orthogonalizes unknown pre‑training gradients to mitigate catastrophic forgetting during continual fine‑tuning of large language models. It generates pseudo data most susceptible to forgetting using a learnable soft prompt with Gumbel‑Softmax, then jointly optimizes model parameters and the prompt via a first‑order Pareto optimizer to enforce orthogonality between new task updates and the estimated gradients. Experiments on multiple LLMs show that EoupCT preserves both task‑specific performance and the models’ inherent general‑purpose knowledge.
arXiv:2609.38929v1 Announce Type: new Abstract: Machine learning systems increasingly face the need to remove the influence of entire data domains, such as toxic language, harmful behavior, or topica...
The paper introduces ADU, a fine‑grained training framework that unlearns sensitive information from large language models by decoupling contextual attention pathways instead of erasing tokens. ADU exploits the distinction between local and global attention heads to identify and suppress attention paths that retrieve persistent sensitive anchors, while preserving local‑attention structure and overall language modeling performance. Evaluation on the TOFU and WMDP benchmarks shows ADU achieves superior forget quality (0.93 on TOFU) and retains 92.9% of model utility compared to 81.9% for existing baselines, with fewer side effects in benign contexts.
arXiv:2608. 16249v1 Announce Type: new Abstract: Machine unlearning in Large Language Models (LLMs) faces a critical trade-off between erasing target knowledge and preserving general utility.
arXiv:2601. 18699v2 Announce Type: replace Abstract: Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities.
Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers...
The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.
arXiv:2606. 23487v2 Announce Type: replace Abstract: Medical vision-language models (VLMs) such as BiomedCLIP generalize broadly, but adapting them to a clinical service is as much a safety problem as an accuracy one.
The paper introduces CLLPU, a multilingual benchmark for evaluating how well large language models can unlearn specific knowledge while controlling its propagation across languages. CLLPU defines two forgetting scenarios—common-goal forgetting, which requires suppression across all languages, and language-conditioned forgetting, which limits suppression to a single language. Using 800 knowledge-unit pairs and 72,000 QA instances in ten languages, the authors test six methods on Llama‑3.1‑8B‑Instruct and find that universal suppression often fails, while language‑specific suppression can unintentionally spread to other languages, highlighting the difficulty of propagation control in multilingual unlearning.