arXiv:2610.27895v1 Announce Type: cross
Abstract: Direct preference optimisation (DPO) is widely used for language model post-training, but its performance depends strongly on the choice of its hyper...
By Hannan Mahadik, David Salinas, Jacek Golebiowski, Aaron Klein
arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.
By Vin\'icius Conte Turani, Ot\'avio Parraga, Jo\~ao Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinsk\"u
The paper introduces SHIFT, a method for unlearning in large language models that avoids explicit forgetting targets by shrinking a fine‑tuned model toward its pre‑fine‑tuning weights and briefly relearning the retain set. In the linearized regime, this shrink‑and‑heal lookahead moves the model a fixed fraction toward the retrained model, and SHIFT extrapolates these per‑token changes into distillation targets for iterative refinement. Evaluated against eleven baselines under the OpenUnlearning protocol, SHIFT achieves the highest aggregate score across all TOFU settings, aligns 3.7× closer to the oracle than the next best method, and preserves the most retain knowledge on MUSE, maintaining proximity to the oracle even under larger optimization budgets and relearning attacks.
By Hoang-Chau Luong, Lingwei Chen
The paper introduces the problem of cross‑lingual loopholes in large language model (LLM) unlearning, where forgetting a fact in one language can leave it accessible in others. It presents a new 174‑language benchmark, the Cross‑Lingual Unlearning Tensor, and proposes COVER, a method that selects a subset of source languages to maximize unlearning coverage under a language budget. Experiments show COVER reduces residual knowledge by 7.8–27.3% compared to uniform selection and works on both synthetic and real low‑resource news data.
By Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
arXiv:2606. 02920v1 Announce Type: new Abstract: Language-model unlearning updates a trained model to behave as if it had not seen selected training examples, while preserving utility and avoiding costly retraining.
By Federico Di Gennaro, Alexander Shevchenko, Fanny Yang
arXiv:2608. 05783v1 Announce Type: cross Abstract: Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs).
By Pawe{\l} Batorski, Przemys{\l}aw Spurek, Paul Swoboda
arXiv:2601. 09172v3 Announce Type: replace Abstract: As Large Language Models (LLMs) increasingly shape online content, removing targeted information from well-trained LLMs (also known as LLM unlearning) has become critical for web governance.
By Pengyang Shao, Naixin Zhai, Lei Chen, Yonghui Yang, Fengbin Zhu, Xun Yang, Meng Wang
CALIBURN is a new approach to large language model (LLM) unlearning that measures a model’s confidence in undesirable knowledge and uses this measure to fine‑tune unlearning gradient updates. By doing so, it offers more precise control over what is forgotten while better preserving the model’s overall utility. Experiments on benchmarks such as MUSE and WMDP show that CALIBURN outperforms existing methods in balancing knowledge removal with utility retention.
By Zhengbang Yang, Yisheng Zhong, Junyuan Hong, Zhuangdi Zhu
RePro is a web‑recycling technique that trains a small language model (as little as 1 B parameters) with reinforcement learning to produce high‑quality, faithful rephrasings of pretraining data. The method uses one quality reward and three faithfulness rewards to preserve core semantics and structure while converting organic data into better training examples. Experiments show that RePro boosts downstream accuracy by 3.7–14.5 % over organic‑only baselines and improves data efficiency 2–3×, outperforming prior prompting‑based recycling approaches.
By Zichun Yu, Chenyan Xiong
The paper explores ways to reduce the memorization of training data in language models, testing three regularizer-based, three finetuning-based, and eleven machine unlearning methods—five of which are newly introduced. It introduces TinyMem, a lightweight suite of small models for rapid testing of these mitigation techniques, and shows that unlearning methods, particularly BalancedSubnet, outperform others in removing memorized content while maintaining task performance. The study also finds that regularizer-based approaches are slow and ineffective, while finetuning methods are costly, especially when high accuracy is required.
By Mansi Sakarvadia, Aswathy Ajith, Arham Khan, Nathaniel Hudson, Caleb Geniesse, Kyle Chard, Yaoqing Yang, Ian Foster, Michael W. Mahoney
arXiv:2609.16229v1 Announce Type: new
Abstract: Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify...
By Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Sijia Liu, Tianlong Chen
arXiv:2602. 14696v2 Announce Type: replace Abstract: Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the target task.
By Nihal V. Nayak, Paula Rodriguez-Diaz, Neha Hulkund, Sara Beery, David Alvarez-Melis