← Back to all news
arXiv AI By Snigdha Paul, Manasi Patwardhan, Arman Cohan

Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning

Read the original on arXiv AI →

The paper introduces the task of Scientific Claim Unlearning and presents a new benchmark, SciUnlearn, to evaluate it. It highlights that language models trained on static scientific corpora risk disseminating outdated or retracted claims as scientific knowledge evolves. Current machine unlearning methods fail to effectively remove claim-level knowledge, often only suppressing it superficially, underscoring the need for specialized techniques for structured knowledge removal.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 28

Graph-Guided Selective Unlearning for Language Models: Controlling Support Routes Beyond Forget Seeds

The paper introduces GRAPHSU, a graph‑guided selective unlearning method for language models that expands deletion beyond explicitly identified forget seeds. By constructing a weighted support‑route graph and propagating deletion pressure, GRAPHSU applies graded forgetting to high‑risk neighboring examples. Experiments on the TOFU and PISTOL benchmarks with GPT‑2 Medium and Llama‑3.2‑3B‑Instruct show that GRAPHSU achieves the lowest utility‑feasible soft leakage, reducing leakage by up to 49.5 percentage points compared to a seed‑only baseline.

By Waqas Khan, Tabinda Sarwar, Jingyue Cong, Xun Yi, Estrid He
llmsfine-tuningbenchmarkssafety
More like this →
arXiv Computation and Language
Aug 25

CALIBURN: Self-Calibrated LLM Unlearning Alignment

CALIBURN is a new approach to large language model (LLM) unlearning that measures a model’s confidence in undesirable knowledge and uses this measure to fine‑tune unlearning gradient updates. By doing so, it offers more precise control over what is forgotten while better preserving the model’s overall utility. Experiments on benchmarks such as MUSE and WMDP show that CALIBURN outperforms existing methods in balancing knowledge removal with utility retention.

By Zhengbang Yang, Yisheng Zhong, Junyuan Hong, Zhuangdi Zhu
llmsbenchmarkssafety
More like this →
arXiv AI
Oct 1

Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning

The paper introduces the problem of cross‑lingual loopholes in large language model (LLM) unlearning, where forgetting a fact in one language can leave it accessible in others. It presents a new 174‑language benchmark, the Cross‑Lingual Unlearning Tensor, and proposes COVER, a method that selects a subset of source languages to maximize unlearning coverage under a language budget. Experiments show COVER reduces residual knowledge by 7.8–27.3% compared to uniform selection and works on both synthetic and real low‑resource news data.

By Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
llmsbenchmarks
More like this →