Hugging Face Trending Papers
Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Sep 11
Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.
arXiv Machine Learning
Jun 5
SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion
arXiv:2605. 07482v2 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining.
arXiv Machine Learning
Oct 19
CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models
The paper introduces CoLU, a confidence‑weighted layerwise unlearning method for large language models that updates only a single transformer layer instead of the entire model. It also proposes Brief Layer Screening (BLS) to quickly rank layers using a normalized forgetting‑retention score, reducing CoLU’s runtime by 60–80% while maintaining performance. Experiments on TOFU, MUSE Books, and News datasets show CoLU outperforms state‑of‑the‑art methods in both forgetting and retention, and uses significantly less GPU memory.
arXiv Machine Learning
Jul 15
Inference-Time Machine Unlearning via Gated Activation Redirection
arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.
arXiv AI
Oct 15
GUARD: Guided Unlearning and Retention via Data Attribution for Large Language Models
arXiv:2506.10946v3 Announce Type: replace-cross Abstract: Unlearning large language models (LLMs) is important due to copyright and privacy concerns. A key challenge is unintended forgetting, where r...
arXiv AI
Sep 24
Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction
arXiv:2609.27355v1 Announce Type: cross Abstract: Unlearning ensures LLM compliance by removing the influence of private or copyrighted training data. However, since LLM models typically undergo post...