← Back to all news
arXiv AI

UNBIND: UNlearning By INference-time Directional Steering for Code LLMs

UNBIND is a code unlearning framework that selectively removes memorized code from large language models at inference time while keeping the model weights unchanged. It constructs separate directional steering for hidden states that correspond to target code, enabling high forgetting rates (97.3–99.1% reduction in target code reproduction) with minimal loss in programming utility. Across multiple baselines, corpora, and evaluation metrics—including F‑BLEU, HumanEval+, and MBPP+—UNBIND consistently achieves the best joint forgetting and utility scores, and it effectively eliminates long exact code spans in repeated extraction tests.

Related stories

arXiv AI
Sep 11

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.

By Ravi Ranjan, Olivera Kotevska, Agoritsa Polyzou
llmsefficiencysafety
More like this →
arXiv AI
3d ago

Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning

The paper introduces the problem of cross‑lingual loopholes in large language model (LLM) unlearning, where forgetting a fact in one language can leave it accessible in others. It presents a new 174‑language benchmark, the Cross‑Lingual Unlearning Tensor, and proposes COVER, a method that selects a subset of source languages to maximize unlearning coverage under a language budget. Experiments show COVER reduces residual knowledge by 7.8–27.3% compared to uniform selection and works on both synthetic and real low‑resource news data.

By Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
llmsbenchmarks
More like this →
arXiv Machine Learning
Sep 16

Test-Time Unlearning via Sparse Autoencoder

arXiv:2609.16229v1 Announce Type: new Abstract: Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify...

By Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Sijia Liu, Tianlong Chen
llmsfine-tuningbenchmarkssafety
More like this →
arXiv AI
3d ago

Mitigating Memorization In Language Models

The paper explores ways to reduce the memorization of training data in language models, testing three regularizer-based, three finetuning-based, and eleven machine unlearning methods—five of which are newly introduced. It introduces TinyMem, a lightweight suite of small models for rapid testing of these mitigation techniques, and shows that unlearning methods, particularly BalancedSubnet, outperform others in removing memorized content while maintaining task performance. The study also finds that regularizer-based approaches are slow and ineffective, while finetuning methods are costly, especially when high accuracy is required.

By Mansi Sakarvadia, Aswathy Ajith, Arham Khan, Nathaniel Hudson, Caleb Geniesse, Kyle Chard, Yaoqing Yang, Ian Foster, Michael W. Mahoney
llmsfine-tuning
More like this →