arXiv Computation and Language August 25, 2026 By Ayush Gupta, Hima Varshini Surisetty, Sreevidya Bollineni, Varad Ingale, Tuhina Tripathi, Abhishek Lalwani, Somya Chatterjee, Sadid Hasan

Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation

Read the original on arXiv Computation and Language →

The Flow has not summarised this story yet — read it at arXiv Computation and Language.

arXiv Machine Learning

Sep 16

Test-Time Unlearning via Sparse Autoencoder

arXiv:2609.16229v1 Announce Type: new Abstract: Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify...

By Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Sijia Liu, Tianlong Chen

llms fine-tuning benchmarks safety

arXiv AI

Sep 1

Measuring the Depth of LLM Unlearning via Activation Patching

arXiv:2605.24614v2 Announce Type: replace-cross Abstract: Large language model (LLM) unlearning has emerged as a crucial post-hoc mechanism for privacy protection and AI safety, yet auditing whether...

By Jaeung Lee, Dohyun Kim, Jaemin Jo

llms benchmarks safety

arXiv AI

Jun 6

LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs

arXiv:2606. 06286v1 Announce Type: cross Abstract: Large language models can reproduce training data, but existing memorization evaluations mostly measure whether models can be forced to do so, rather than whether they do so under ordinary use.

By Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke Poech

llms safety

arXiv Machine Learning

Sep 30

Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix it

arXiv:2609.36612v1 Announce Type: new Abstract: Unlearning in large language models (LLMs) is typically evaluated at the output level, where a model appears to suppress sensitive or undesirable conte...

By Hadi Reisizadeh, Jiajun Ruan, Sijia Liu, Mingyi Hong

llms benchmarks safety

arXiv AI

Sep 1

Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning

The paper introduces FAB, an attack that uses meta‑learning to embed dormant adversarial behaviors into large language models (LLMs). These behaviors remain inactive until the model is finetuned by downstream users, at which point the model can exhibit unwanted actions such as unsolicited advertising, jailbreakability, or over‑refusal. FAB is shown to be effective across multiple LLMs and resilient to various finetuning settings.

By Thibaud Gloaguen, Mark Vero, Robin Staab, Martin Vechev

llms reinforcement-learning fine-tuning efficiency safety

arXiv Machine Learning

Jun 16

Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget

arXiv:2606. 16110v1 Announce Type: new Abstract: Machine unlearning has been extensively studied in response to growing privacy concerns and regulatory requirements.

By Dayong Ye, Tianqing Zhu, Ruiding Huang, Xinbo Fu, Jiayang Li, Bo Liu, Huan Huo, Wanlei Zhou

llms fine-tuning safety

One email a morning, machine-written

Related stories

Test-Time Unlearning via Sparse Autoencoder

Measuring the Depth of LLM Unlearning via Activation Patching

LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs

Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix it

Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning

Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget