arXiv Machine Learning

SAUL: Sharpness-Aware Augmented-Lagrangian Unlearning

arXiv:2608. 16249v1 Announce Type: new Abstract: Machine unlearning in Large Language Models (LLMs) faces a critical trade-off between erasing target knowledge and preserving general utility.

arXiv Machine Learning

Jun 29

Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting

arXiv:2510. 18874v3 Announce Type: replace Abstract: Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities -- a phenomenon classically known as catastrophic forgetting.

By Howard Chen, Noam Razin, Karthik Narasimhan, Danqi Chen

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 5

Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance

arXiv:2606. 06320v1 Announce Type: new Abstract: Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities.

By Gizem Y\"uce, Giorgos Nikolaou, Nicolas Flammarion

llms

arXiv Machine Learning

Jul 7

RL Forgets! Towards Continual Policy Optimization

arXiv:2607. 04364v1 Announce Type: new Abstract: Continual post-training is becoming a central paradigm for adapting vision-language models to evolving tasks.

By Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou, Bo Ye, Jian Zhao, Min-Ling Zhang, Tong Wei

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 10

Null-Space Constrained Low-Rank Adaptation for Response-Specified Large Language Model Unlearning

arXiv:2606. 10989v1 Announce Type: new Abstract: Large language model unlearning aims to suppress designated undesirable knowledge while preserving benign capabilities.

By Bocheng Ju, Jianhua Wang, Chengliang Liu, Xiaolin Chang

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jun 16

Replay What Matters: Off-Policy Replay for Efficient LLM Reinforcement Unlearning

arXiv:2606. 15333v1 Announce Type: cross Abstract: LLM unlearning has emerged as a cost-effective alternative to full retraining for removing hazardous knowledge from pretrained models while preserving general utility.

By Zirui Pang, Chenlong Zhang, Haosheng Tan, Zhuoran Jin, Jiaheng Wei, Zixin Zhong

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jul 15

Inference-Time Machine Unlearning via Gated Activation Redirection

arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.

By Vin\'icius Conte Turani, Ot\'avio Parraga, Jo\~ao Vitor Boer Abitante, Kristen K. Arguello, Joana Pasquali, Ramiro N. Barros, Flavio du Pin Calmon, Christian Mattjie, Rodrigo C. Barros, Lucas S. Kupssinsk\"u

llms efficiency safety

Hugging Face Trending Papers

Jun 9

Null-Space Constrained Low-Rank Adaptation for Response-Specified Large Language Model Unlearning

Large language model unlearning aims to suppress designated undesirable knowledge while preserving benign capabilities. Many unlearning objectives focus on suppressing undesired answers, while recent target-guided variants specify replacement behavior but still leave update locality largely unconstrained.

llms fine-tuning benchmarks safety

arXiv AI

Aug 7

GROM: Gradient-Free Rapid One-Shot Machine Unlearning

arXiv:2608. 05783v1 Announce Type: cross Abstract: Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs).

By Pawe{\l} Batorski, Przemys{\l}aw Spurek, Paul Swoboda

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 2

Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization

arXiv:2606. 00132v1 Announce Type: cross Abstract: While finetuning effectively adapts foundation models to specialized downstream tasks, it can degrade nontarget capabilities acquired during pretraining.

By Dongjun Kim, Adrian de Wynter, Huancheng Chen, Heasung Kim, Haris Vikalo

fine-tuning

arXiv Machine Learning

Jun 2

Multi-Objective Reference-Aligned Machine Unlearning

arXiv:2606. 00399v1 Announce Type: new Abstract: Machine unlearning aims to remove the influence of specific training samples while preserving the model's utility.

By Rasa Khosrowshahli, Stephen Asobiela, Beatrice Ombuki-Berman, Shahryar Rahnamayan

safety

arXiv Machine Learning

Jul 3

Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation

arXiv:2602. 00722v2 Announce Type: replace Abstract: Parameter-efficient continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge.

By Hao Gu, Mao-Lin Luo, Zi-Hao Zhou, Han-Chen Zhang, Min-Ling Zhang, Tong Wei

llms multimodal

arXiv AI

Jul 13

Interference and Retention in Continual Learning

arXiv:2607. 09202v1 Announce Type: cross Abstract: Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation.

By Julius St\"ork

efficiency benchmarks

Related stories