Nullify is a training‑free method for unlearning sensitive information from large language models by steering activations during inference. It uses steering vectors to redirect privacy‑related activations away from memorized answers while enforcing a null‑space constraint that preserves activations for retained queries, thereby maintaining model utility. Experiments on TOFU and MUSE demonstrate that Nullify matches or outperforms existing baselines in forget quality while keeping utility nearly intact, all without any weight updates.
Large Language Models (LLMs) inevitably internalize substantial amounts of sensitive or private information during pre-training, while LLM unlearning aims to selectively erase specific knowledge to pr...
arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.
arXiv:2606. 06320v1 Announce Type: new Abstract: Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities.
The paper identifies a problem in large language model (LLM) unlearning called forget‑set misalignment, where the set of data to be forgotten does not match what the model has actually memorized. Two failure modes are described: Under Unlearning, where memorized information is omitted from the forget set, and Out‑of‑Knowledge Unlearning, where the algorithm attempts to forget knowledge the model never learned, harming performance. The authors propose CONfs, a data‑blind framework that constructs model‑aligned forget sets by eliciting the model’s memorized knowledge, and demonstrate that it achieves near‑gold standard forgetting while preserving utility better than other data‑blind methods.
Machine unlearning for large language models (LLMs) often assumes that a pre-defined forget set matches what the model has memorized, but this frequently breaks in realistic privacy settings where the...
Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers...
The paper introduces ADU, a fine‑grained training framework that unlearns sensitive information from large language models by decoupling contextual attention pathways instead of erasing tokens. ADU exploits the distinction between local and global attention heads to identify and suppress attention paths that retrieve persistent sensitive anchors, while preserving local‑attention structure and overall language modeling performance. Evaluation on the TOFU and WMDP benchmarks shows ADU achieves superior forget quality (0.93 on TOFU) and retains 92.9% of model utility compared to 81.9% for existing baselines, with fewer side effects in benign contexts.
arXiv:2609.05966v1 Announce Type: new Abstract: Existing unlearning approaches typically rely on post hoc weight adaptation or distillation, leading to duplicated memory costs, degraded generalizatio...
arXiv:2512. 05254v2 Announce Type: replace Abstract: As concerns around data privacy in machine learning grow, the ability to unlearn, or remove, specific data points from trained models becomes increasingly important.
The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.
arXiv:2507. 04219v5 Announce Type: replace-cross Abstract: Current unlearning methods for LLMs optimize on the private information they seek to remove by incorporating it into their fine-tuning data.
arXiv:2508.20443v3 Announce Type: replace Abstract: Large language models (LLMs) are trained on massive datasets that may include private or copyrighted content. Due to growing privacy and ownership...