← Back to all news
arXiv Computation and Language By Tyler Lizzo, Larry Heck

Behavioral Forgetting Is Not Privacy in Machine Unlearning

Read the original on arXiv Computation and Language →

The paper argues that machine unlearning, which seeks to erase specific information from large language models, is often evaluated only on a single benchmark distribution, leading to an incomplete assessment of privacy. The authors demonstrate that forgetting can differ significantly when the same information is queried in different languages and scripts across multilingual models, revealing a gap between behavioral forgetting and true inaccessibility. They call for broader evaluation that includes access‑path generalization and adversarial recoverability to ensure genuine privacy protection.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Sep 2

Confess What You Know: Forget-Set Misalignment with Model Knowledge in LLM Unlearning

The paper identifies a problem in large language model (LLM) unlearning called forget‑set misalignment, where the set of data to be forgotten does not match what the model has actually memorized. Two failure modes are described: Under Unlearning, where memorized information is omitted from the forget set, and Out‑of‑Knowledge Unlearning, where the algorithm attempts to forget knowledge the model never learned, harming performance. The authors propose CONfs, a data‑blind framework that constructs model‑aligned forget sets by eliciting the model’s memorized knowledge, and demonstrate that it achieves near‑gold standard forgetting while preserving utility better than other data‑blind methods.

By Miso Kim, Georu Lee, Seungwon Jeong, Woojin Lee
llmsmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 9

Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning

Nullify is a training‑free method for unlearning sensitive information from large language models by steering activations during inference. It uses steering vectors to redirect privacy‑related activations away from memorized answers while enforcing a null‑space constraint that preserves activations for retained queries, thereby maintaining model utility. Experiments on TOFU and MUSE demonstrate that Nullify matches or outperforms existing baselines in forget quality while keeping utility nearly intact, all without any weight updates.

By Wei Zhai, Xiang Liu, Qiang Huang, Rui Qian, Lemao Liu, Ziwei Li, Ziqi Wang, Zhitao Huang, Dejing Dou
llmsfine-tuningsafety
More like this →
arXiv AI
Oct 1

Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning

The paper introduces the problem of cross‑lingual loopholes in large language model (LLM) unlearning, where forgetting a fact in one language can leave it accessible in others. It presents a new 174‑language benchmark, the Cross‑Lingual Unlearning Tensor, and proposes COVER, a method that selects a subset of source languages to maximize unlearning coverage under a language budget. Experiments show COVER reduces residual knowledge by 7.8–27.3% compared to uniform selection and works on both synthetic and real low‑resource news data.

By Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
llmsbenchmarks
More like this →