arXiv Machine Learning

Not All Change Is Damage: Update-Matched Unlearning for Large Language Models

The paper introduces Update‑Matched Unlearning (UMU), a method that applies the same simulated relearning update to both a defended model and an undefended reference to better preserve plasticity while resisting relearning of deleted knowledge. Experiments on Llama and Qwen models with TOFU and FaithUn datasets show that UMU reduces harmful and benign relearning by 30.4 and 9.9 points on average and increases new‑task accuracy by 3.5 points, supporting the reference‑timing hypothesis. The study demonstrates that robust unlearning should selectively prevent recovery of deleted knowledge without suppressing the model’s ability to learn new tasks.

arXiv AI

Sep 11

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.

By Ravi Ranjan, Olivera Kotevska, Agoritsa Polyzou

llms efficiency safety

Hugging Face Trending Papers

Sep 9

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers...

llms efficiency safety

arXiv Machine Learning

Sep 30

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

arXiv:2609.37076v1 Announce Type: new Abstract: Large language models trained on vast corpora inherently risk memorizing harmful content that may later re-emerge in their outputs. To mitigate this is...

By Puning Yang, Qizhou Wang, Junchi Yu, Bo Han, Xiuying Chen

llms

arXiv Machine Learning

Sep 16

Test-Time Unlearning via Sparse Autoencoder

arXiv:2609.16229v1 Announce Type: new Abstract: Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify...

By Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Sijia Liu, Tianlong Chen

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jul 15

Inference-Time Machine Unlearning via Gated Activation Redirection

arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.

By Vin\'icius Conte Turani, Ot\'avio Parraga, Jo\~ao Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinsk\"u

llms efficiency safety

arXiv Machine Learning

23h ago

Binding-Specific Relearning Savings for Test-Time Continual Learning

arXiv:2610.22381v1 Announce Type: new Abstract: When a fact becomes harder to elicit from a language model, has the model lost it, or has it merely become less accessible? Parameter-updating test-tim...

By Zhe Zhou

llms agents fine-tuning

arXiv Machine Learning

Jun 5

SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

arXiv:2605. 07482v2 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining.

By Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei, Mohammad Rostami, Jesse Thomason, Robin Jia

llms efficiency benchmarks

arXiv Machine Learning

Jul 22

Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs

arXiv:2607. 18639v1 Announce Type: new Abstract: Safety interventions on dual-use knowledge typically choose between destroying hazardous content (e.

By Seunghyun Lee, Dongyoon Han, Sangdoo Yun

llms fine-tuning benchmarks safety

arXiv Computation and Language

Oct 9

Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning

The paper introduces Propose-Then-Project Unlearning (PTP-U), a method for large language model unlearning that first weakens target knowledge associations and then projects non-target outputs back to the original distribution to recover capabilities. PTP-U aims to satisfy forgetting constraints while preserving fluent generation and performance on non-target tasks. Experiments on three benchmarks show PTP-U achieves a strong forgetting‑retention trade‑off, with 81.22%–91.03% forgetting and an average of 94.20% non‑target utility.

By Xunlei Chen, Qinghui Gong, Jingkun Xue, Qihe Liu, Shijie Zhou, Fei Ye

llms benchmarks

arXiv AI

4d ago

GUARD: Guided Unlearning and Retention via Data Attribution for Large Language Models

arXiv:2506.10946v3 Announce Type: replace-cross Abstract: Unlearning large language models (LLMs) is important due to copyright and privacy concerns. A key challenge is unintended forgetting, where r...

By Peizhi Niu, Evelyn Ma, Boya Hou, Huiting Zhou, Duo Zhou, Huan Zhang, S. Rasoul Etesami, Olgica Milenkovic

llms benchmarks safety

Hugging Face Trending Papers

Jul 21

Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs

Safety interventions on dual-use knowledge typically choose between destroying hazardous content (e. g.

llms fine-tuning benchmarks safety

arXiv Machine Learning

23h ago

CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models

arXiv:2610.22721v1 Announce Type: cross Abstract: Large language model (LLM) unlearning aims to forget certain information while preserving others, without retraining from scratch. SoTA methods such...

By Jingkai Guo, Ziyi Zhao, Chaitali Chakrabarti, Deliang Fan

llms fine-tuning benchmarks

Related stories