arXiv AI

Measuring the Depth of LLM Unlearning via Activation Patching

arXiv Computation and Language
Aug 25

Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation

arXiv:2608.21606v1 Announce Type: new Abstract: Machine unlearning aims to remove the influence of targeted training data from a model while preserving its remaining capabilities, but evaluating whet...

By Ayush Gupta, Hima Varshini Surisetty, Sreevidya Bollineni, Varad Ingale, Tuhina Tripathi, Abhishek Lalwani, Somya Chatterjee, Sadid Hasan
arXiv AI
Oct 15

Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning

arXiv:2610.10358v2 Announce Type: replace Abstract: As multimodal large language models (MLLMs) become more capable and widely deployed, concerns about privacy and safety have become increasingly pre...

By Junkai Chen, Yuhao He, Qianshan Wei, Junxiang You, Jingwen Shao, Junkai Lin, Zhongkai Yue, Xiaotian Ye, Zhengbo Jiao, Jiali Cheng, Zhijie Deng, Kening Zheng, Ruiqi Liu, Hadi Amiri, Yi Yu, Zhenan Sun, Qi Li, Ka-Ho Chow, Sijia Liu, Liang Wang, Jiaqi Li, Shu Wu
arXiv AI
Oct 16

Evaluating Model Capabilities After Benchmark Saturation

The paper examines whether transcripts of model solutions can reveal capabilities after benchmark scores saturate. By simulating saturation with historical data, the authors test protocols that use only correctly solved transcripts to predict performance on unsaturated items. Pairwise judging of transcripts outperforms other methods, reducing prediction error by 5.3×, though its effectiveness can vary across distribution shifts.

By Morio Hamada, Jehyeok Yeon, Matteo Migliarini, Justin Olive, Magda Dubois
arXiv Computation and Language
Aug 25

CALIBURN: Self-Calibrated LLM Unlearning Alignment

CALIBURN is a new approach to large language model (LLM) unlearning that measures a model’s confidence in undesirable knowledge and uses this measure to fine‑tune unlearning gradient updates. By doing so, it offers more precise control over what is forgotten while better preserving the model’s overall utility. Experiments on benchmarks such as MUSE and WMDP show that CALIBURN outperforms existing methods in balancing knowledge removal with utility retention.

By Zhengbang Yang, Yisheng Zhong, Junyuan Hong, Zhuangdi Zhu