Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Current unlearning benchmarks include mainly single-hop questions and a narrow set of multi-hop questions.
The paper introduces GONE, a benchmark for evaluating knowledge unlearning in large language models using structured knowledge graphs, and presents Neighborhood-Expanded Distribution Shaping (NEDS), a framework that leverages graph connectivity to separate forgotten facts from their semantic neighborhood. GONE disentangles direct fact removal, reasoning-based leakage, and catastrophic forgetting, while NEDS achieves high unlearning efficacy and locality on LLaMA-3-8B and Mistral-7B. The dataset is publicly available on Hugging Face.
By Chahana Dahal, Ashutosh Balasubramaniam, Zuobin Xiong
arXiv:2608.21606v1 Announce Type: new
Abstract: Machine unlearning aims to remove the influence of targeted training data from a model while preserving its remaining capabilities, but evaluating whet...
By Ayush Gupta, Hima Varshini Surisetty, Sreevidya Bollineni, Varad Ingale, Tuhina Tripathi, Abhishek Lalwani, Somya Chatterjee, Sadid Hasan
The paper introduces GRAPHSU, a graph‑guided selective unlearning method for language models that expands deletion beyond explicitly identified forget seeds. By constructing a weighted support‑route graph and propagating deletion pressure, GRAPHSU applies graded forgetting to high‑risk neighboring examples. Experiments on the TOFU and PISTOL benchmarks with GPT‑2 Medium and Llama‑3.2‑3B‑Instruct show that GRAPHSU achieves the lowest utility‑feasible soft leakage, reducing leakage by up to 49.5 percentage points compared to a seed‑only baseline.
By Waqas Khan, Tabinda Sarwar, Jingyue Cong, Xun Yi, Estrid He
arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.
By Amit Peleg, Naman Deep Singh, Naama Pearl, Bibhabasu Mohapatra, Matthias Hein
arXiv:2608.22527v2 Announce Type: replace
Abstract: Recently, machine unlearning, the removal of specific training data influence from a model, has gained increasing attention. In large language mode...
By Noam Diamant, Neta Glazer, Ethan Fetaya