arXiv:2602. 09234v2 Announce Type: replace-cross Abstract: Continual learning has become a trending topic in machine learning.
arXiv:2608. 01475v1 Announce Type: new Abstract: Neural networks that can grow or both grow and shrink during learning, referred to as growing neural networks and elastic neural networks, respectively, have recently been explored in offline continual learning with a particular focus on catastrophic forgetting.
The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning. Although this phenomenon has been known for decades, it has mostly been studied in older, relatively small architectures and rarely in natural-language domains.
arXiv:2606. 24752v1 Announce Type: new Abstract: The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning.
The paper investigates continual machine unlearning, where models must forget data over time. It identifies a fundamental issue called plasticity collapse, where successive unlearning requests cause geometric constraints that saturate parameter space, leading to two failure modes: forward failure (reduced forgetting quality) and backward failure (re‑memorization). Experiments across architectures and datasets confirm that plasticity collapse is a pervasive problem in continual unlearning.
arXiv:2607. 05609v1 Announce Type: cross Abstract: The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting.
The paper introduces SingularClip, a method that periodically clips the singular values of all weight matrices to prevent spectral collapse, a newly identified cause of plasticity loss in neural networks trained on nonstationary tasks. The authors empirically and theoretically analyze how growing anisotropy of singular values degrades the ability to fit new targets, and demonstrate that SingularClip outperforms baseline approaches in both continual supervised learning and deep reinforcement learning settings.
arXiv:2608. 06216v1 Announce Type: cross Abstract: Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.
The paper investigates continual reinforcement learning using neuroevolution, comparing evolution strategies (ES) and genetic algorithms (GAs) across diverse environments and network sizes. ES consistently achieves a better balance between stability and plasticity, while GAs are more plastic but forget more. The authors attribute this to ES finding wider neighborhoods in weight space, with overlap between consecutive tasks correlating with the stability-plasticity trade‑off, and note that common RL plasticity issues do not transfer to neuroevolution.
arXiv:2607. 13432v1 Announce Type: new Abstract: Plasticity -- a neural network's ability to adapt to new tasks -- is critical for continual and transfer learning.
arXiv:2609.36081v1 Announce Type: new Abstract: Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure th...
arXiv:2606. 09762v1 Announce Type: cross Abstract: Continual training of deep neural networks under non-stationarity often leads to a progressive loss of plasticity, eventually limiting further learning.
arXiv:2608.23889v1 Announce Type: cross Abstract: Plasticity under changing environments is central to both evolutionary biology and continual learning. Motivated by recent work on genotype--phenotyp...