arXiv:2606. 09762v1 Announce Type: cross Abstract: Continual training of deep neural networks under non-stationarity often leads to a progressive loss of plasticity, eventually limiting further learning.
By Andries Rosseau, Robert M\"uller, Ann Now\'e
arXiv:2608. 01475v1 Announce Type: new Abstract: Neural networks that can grow or both grow and shrink during learning, referred to as growing neural networks and elastic neural networks, respectively, have recently been explored in offline continual learning with a particular focus on catastrophic forgetting.
By Jeong Min Kong, Richard S. Sutton
arXiv:2607. 13432v1 Announce Type: new Abstract: Plasticity -- a neural network's ability to adapt to new tasks -- is critical for continual and transfer learning.
By Jiaxuan Cheng
arXiv:2608. 12874v1 Announce Type: new Abstract: Plasticity loss has emerged as a critical challenge in continual learning that significantly hinders the acquisition of sequential tasks.
By Zeyang Zhang, Tieliang Gong, Junyan Lu, Weizhan Zhang
Plasticity loss has emerged as a critical challenge in continual learning that significantly hinders the acquisition of sequential tasks. While optimizing activation designs offers a potential solution, current fixed-form functions suffer from an inherent spectral bias towards low-frequency variations, whereas learnable variants permit unconstrained updates that induce catastrophic forgetting.
arXiv:2606. 30559v1 Announce Type: new Abstract: We characterize weakly regularized continual classification in homogeneous models as sequential projections onto task margin sets.
By Matan Schliserman, Gon Buzaglo, Itay Evron, Daniel Soudry