arXiv AI August 25, 2026 By Xunlei Chen, Qirui Ye, Yuang Li, Yi Gong, Zhaokun Wang, Wenyi Li, Shiyao Guo, Jinyu Guo

Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality

The paper introduces ADU, a fine‑grained training framework that unlearns sensitive information from large language models by decoupling contextual attention pathways instead of erasing tokens. ADU exploits the distinction between local and global attention heads to identify and suppress attention paths that retrieve persistent sensitive anchors, while preserving local‑attention structure and overall language modeling performance. Evaluation on the TOFU and WMDP benchmarks shows ADU achieves superior forget quality (0.93 on TOFU) and retains 92.9% of model utility compared to 81.9% for existing baselines, with fewer side effects in benign contexts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 5

Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance

arXiv:2606. 06320v1 Announce Type: new Abstract: Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities.

By Gizem Y\"uce, Giorgos Nikolaou, Nicolas Flammarion

llms

arXiv Computation and Language

5d ago

Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning

The paper introduces Propose-Then-Project Unlearning (PTP-U), a method for large language model unlearning that first weakens target knowledge associations and then projects non-target outputs back to the original distribution to recover capabilities. PTP-U aims to satisfy forgetting constraints while preserving fluent generation and performance on non-target tasks. Experiments on three benchmarks show PTP-U achieves a strong forgetting‑retention trade‑off, with 81.22%–91.03% forgetting and an average of 94.20% non‑target utility.

By Xunlei Chen, Qinghui Gong, Jingkun Xue, Qihe Liu, Shijie Zhou, Fei Ye

llms benchmarks

arXiv Computation and Language

6d ago

A Deafening Silence: Catastrophic Forgetting Lives in the Output Embeddings of Tokens the Data Never Speaks

arXiv:2610.09835v1 Announce Type: new Abstract: Continual pre-training and fine-tuning in Large Language Models (LLMs) inevitably induce catastrophic forgetting, typically mitigated by replay using o...

By Jonghyun Han, Younghoon Song, Jongyoul Park

llms rag fine-tuning

arXiv Machine Learning

5d ago

Remember to Forget: Gated Adaptive Positional Encoding

The paper introduces GAPE (Gated Adaptive Positional Encoding), a drop‑in augmentation for rotary positional encodings that adds content‑aware bias to attention logits while preserving rotary geometry. GAPE uses a query‑dependent gate to contract irrelevant context and a key‑dependent gate to keep salient distant tokens, leading to exponential attenuation of weakly protected context. Experiments show that GAPE improves long‑context robustness, extrapolating up to eight times the training length, and can be retrofitted into a pretrained 7B model without losing benchmark performance.

By Riccardo Ali, Alessio Borgi, Mario Severino, Pietro Li\`o, Christopher Irwin

llms benchmarks safety

arXiv Machine Learning

Sep 16

Test-Time Unlearning via Sparse Autoencoder

arXiv:2609.16229v1 Announce Type: new Abstract: Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify...

By Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Sijia Liu, Tianlong Chen

llms fine-tuning benchmarks safety

arXiv AI

Jun 29

Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

arXiv:2511. 20196v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) can inadvertently memorize privacy-sensitive information during training.

By Zhen Zeng, Leijiang Gu, Zhangling Duan, Feng Li, Cees G. M. Snoek, Meng Wang, Zenglin Shi

llms fine-tuning multimodal benchmarks safety

One email a morning, machine-written

Related stories

Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance

Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning

A Deafening Silence: Catastrophic Forgetting Lives in the Output Embeddings of Tokens the Data Never Speaks

Remember to Forget: Gated Adaptive Positional Encoding

Test-Time Unlearning via Sparse Autoencoder

Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning