arXiv:2607. 05609v1 Announce Type: cross Abstract: The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting.
By Giulia Lanzillotta, Mandana Samiei, Doina Precup, Razvan Pascanu, Claire Vernade
The paper introduces EoupCT, a framework that estimates and orthogonalizes unknown pre‑training gradients to mitigate catastrophic forgetting during continual fine‑tuning of large language models. It generates pseudo data most susceptible to forgetting using a learnable soft prompt with Gumbel‑Softmax, then jointly optimizes model parameters and the prompt via a first‑order Pareto optimizer to enforce orthogonality between new task updates and the estimated gradients. Experiments on multiple LLMs show that EoupCT preserves both task‑specific performance and the models’ inherent general‑purpose knowledge.
By Bing Wang, Changchun Li, Xin-Qiang Cai, Lin Yuanbo Wu, Ximing Li, Gang Niu, Masashi Sugiyama
arXiv:2608.27518v1 Announce Type: new
Abstract: Continual learning (CL) and model merging (MM) both aim to obtain a single model that performs well across multiple tasks, challenged respectively by c...
By Shangge Liu, Yuehan Yin, Yinghuan Shi, Lei Wang, Wenbin Li
arXiv:2606. 29832v1 Announce Type: new Abstract: Machine unlearning aims to eliminate the influence of specific data from trained models to safeguard privacy.
By Yiting Hu, Lingjie Duan, Qian Zhang
arXiv:2604.13460v2 Announce Type: replace-cross
Abstract: A central challenge in continual learning is forgetting: the loss of performance on previously learned tasks after learning new ones. Prior t...
By Zonghuan Xu, Xingjun Ma
arXiv:2510. 18874v3 Announce Type: replace Abstract: Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities -- a phenomenon classically known as catastrophic forgetting.
By Howard Chen, Noam Razin, Karthik Narasimhan, Danqi Chen