← Back to all news
arXiv AI By Waqas Khan, Tabinda Sarwar, Jingyue Cong, Xun Yi, Estrid He

Graph-Guided Selective Unlearning for Language Models: Controlling Support Routes Beyond Forget Seeds

Read the original on arXiv AI →

The paper introduces GRAPHSU, a graph‑guided selective unlearning method for language models that expands deletion beyond explicitly identified forget seeds. By constructing a weighted support‑route graph and propagating deletion pressure, GRAPHSU applies graded forgetting to high‑risk neighboring examples. Experiments on the TOFU and PISTOL benchmarks with GPT‑2 Medium and Llama‑3.2‑3B‑Instruct show that GRAPHSU achieves the lowest utility‑feasible soft leakage, reducing leakage by up to 49.5 percentage points compared to a seed‑only baseline.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 3

GONE: Structural Knowledge Unlearning via Neighborhood-Expanded Distribution Shaping

The paper introduces GONE, a benchmark for evaluating knowledge unlearning in large language models using structured knowledge graphs, and presents Neighborhood-Expanded Distribution Shaping (NEDS), a framework that leverages graph connectivity to separate forgotten facts from their semantic neighborhood. GONE disentangles direct fact removal, reasoning-based leakage, and catastrophic forgetting, while NEDS achieves high unlearning efficacy and locality on LLaMA-3-8B and Mistral-7B. The dataset is publicly available on Hugging Face.

By Chahana Dahal, Ashutosh Balasubramaniam, Zuobin Xiong
llmsfine-tuningefficiencybenchmarkssafety
More like this →
arXiv Computation and Language
Aug 25

CALIBURN: Self-Calibrated LLM Unlearning Alignment

CALIBURN is a new approach to large language model (LLM) unlearning that measures a model’s confidence in undesirable knowledge and uses this measure to fine‑tune unlearning gradient updates. By doing so, it offers more precise control over what is forgotten while better preserving the model’s overall utility. Experiments on benchmarks such as MUSE and WMDP show that CALIBURN outperforms existing methods in balancing knowledge removal with utility retention.

By Zhengbang Yang, Yisheng Zhong, Junyuan Hong, Zhuangdi Zhu
llmsbenchmarkssafety
More like this →
arXiv Machine Learning
1d ago

When Forgotten Facts Return: Relearning-Aware Unlearning of Knowledge Graph Embeddings via Tangent Kernels

The paper introduces Kernel-Anticipated Unlearning (KAU), a method for approximate unlearning of knowledge-graph embeddings that accounts for relearning during continued training. KAU uses the tangent kernel to model how forgotten facts’ scores change, expressing this response in closed form and guiding edits to minimize future leakage. Experiments show KAU reduces leakage by 46‑63% on FB15k-237 and 50‑82% on CoDEx-L compared to the strongest existing unlearner, while preserving higher utility.

By Parnian Fazel, Mohammad M Maheri, Hamed Haddadi
ragbenchmarks
More like this →