arXiv AI September 11, 2026 By Ravi Ranjan, Olivera Kotevska, Agoritsa Polyzou

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Sep 9

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers...

llms efficiency safety

arXiv Machine Learning

Oct 19

CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models

The paper introduces CoLU, a confidence‑weighted layerwise unlearning method for large language models that updates only a single transformer layer instead of the entire model. It also proposes Brief Layer Screening (BLS) to quickly rank layers using a normalized forgetting‑retention score, reducing CoLU’s runtime by 60–80% while maintaining performance. Experiments on TOFU, MUSE Books, and News datasets show CoLU outperforms state‑of‑the‑art methods in both forgetting and retention, and uses significantly less GPU memory.

By Jingkai Guo, Ziyi Zhao, Chaitali Chakrabarti, Deliang Fan

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 5

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

arXiv:2605. 07482v2 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining.

By Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei, Mohammad Rostami, Jesse Thomason, Robin Jia

llms efficiency benchmarks

arXiv Machine Learning

Jul 15

Inference-Time Machine Unlearning via Gated Activation Redirection

arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.

By Vin\'icius Conte Turani, Ot\'avio Parraga, Jo\~ao Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinsk\"u

llms efficiency safety

arXiv AI

Sep 24

Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction

arXiv:2609.27355v1 Announce Type: cross Abstract: Unlearning ensures LLM compliance by removing the influence of private or copyrighted training data. However, since LLM models typically undergo post...

By Jialu Wang, Jianing Deng, Shuqing Luo, Yuanzhe Li, Dongwei Wang, Jingtong Hu, Huanrui Yang, Song Wang, Tianlong Chen

llms efficiency benchmarks

arXiv AI

Oct 15

GUARD: Guided Unlearning and Retention via Data Attribution for Large Language Models

arXiv:2506.10946v3 Announce Type: replace-cross Abstract: Unlearning large language models (LLMs) is important due to copyright and privacy concerns. A key challenge is unintended forgetting, where r...

By Peizhi Niu, Evelyn Ma, Boya Hou, Huiting Zhou, Duo Zhou, Huan Zhang, S. Rasoul Etesami, Olgica Milenkovic

llms benchmarks safety

One email a morning, machine-written

Related stories

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

Inference-Time Machine Unlearning via Gated Activation Redirection

Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction

GUARD: Guided Unlearning and Retention via Data Attribution for Large Language Models