The paper investigates continual machine unlearning, where models must forget data over time. It identifies a fundamental issue called plasticity collapse, where successive unlearning requests cause geometric constraints that saturate parameter space, leading to two failure modes: forward failure (reduced forgetting quality) and backward failure (re‑memorization). Experiments across architectures and datasets confirm that plasticity collapse is a pervasive problem in continual unlearning.
By Yingdan Shi, Xiang Xu, Kaize Ding, Alfred O. Hero, Ren Wang
arXiv:2608. 06216v1 Announce Type: cross Abstract: Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.
By Zhiyan Hou, Dan Zhang, Tao Feng, Liyuan Wang, Wei Li, Xiangzhao Hao, Hongyan An, Junfeng Fang, Haokai Ma, Zhaohui Xu, Haiyun Guo, Jinqiao Wang, Tat-Seng Chua
arXiv:2602. 03846v2 Announce Type: replace-cross Abstract: We develop a continual learning method for pretrained models that \emph{requires no access to old-task data}, addressing a practical barrier in foundation model adaptation where pretraining distributions are often unavailable.
By Romain Cosentino
arXiv:2604. 27031v2 Announce Type: replace-cross Abstract: In a continual learning setting, we require a model to be plastic enough to learn a new task and stable enough to not disturb previously learned capabilities.
By Karthik Charan Raghunathan, Christian Metzner, Laura Kriener, Melika Payvand
arXiv:2608. 01475v1 Announce Type: new Abstract: Neural networks that can grow or both grow and shrink during learning, referred to as growing neural networks and elastic neural networks, respectively, have recently been explored in offline continual learning with a particular focus on catastrophic forgetting.
By Jeong Min Kong, Richard S. Sutton
arXiv:2609.38911v1 Announce Type: new
Abstract: Learning must acquire new capabilities while preserving both prior responsibilities and the capacity to learn again. We introduce Generalized Residual...
By Dongxu Li, Yinuo Zhang, Hongyu Zhang, Feng Tian
arXiv:2607. 25531v1 Announce Type: cross Abstract: Contemporary machine learning struggles to learn continually, reuse prior knowledge, and expose a comprehensible internal structure.
By Zeki Doruk Erden
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
By Tieliang Gong, Zhongbo Zhang, Wen Wen, Yong-Jin Liu
arXiv:2601. 22012v3 Announce Type: replace Abstract: Catastrophic forgetting in continual learning is often measured at the performance or last-layer representation level, overlooking the underlying mechanisms.
By Sergi Masip, Gido M. van de Ven, Javier Ferrando, Tinne Tuytelaars
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
By Anne Harrington, Nayan Saxena, Michael Murphy, Anastasia Borovykh, Zeyu Yun, Sridhar Kamath, Ara Eindra Kyi, Trevor Darrell, Jitendra Malik, Yutong Bai
The paper argues that catastrophic forgetting and loss of plasticity alone cannot explain why naive sequential training underperforms offline joint training. It introduces data co-observation as a third factor, showing that observing training data together consistently improves performance across supervised and self-supervised settings. The study also reinterprets common continual learning methods, suggesting that memory replay’s success stems from restoring co-observation benefits rather than merely mitigating forgetting.
By Timm Hess, Abhishek Jha, Gido M. van de Ven, Tinne Tuytelaars
arXiv:2609.36375v1 Announce Type: new
Abstract: Continual learning is usually studied through mechanisms that preserve old knowledge. We develop Successional Learning Theory (SLT), a mesoscopic accou...
By Shoaib Ahmed Dipu, Md Salman Shamil, Sayeed Shafayet Chowdhury