arXiv AI
Jul 8
To Retain or to Adapt? Generalizing Continual Learning
arXiv:2607. 05609v1 Announce Type: cross Abstract: The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting.
Plasticity of Growing and Elastic Neural Networks in Online Continual Learning
arXiv:2608. 01475v1 Announce Type: new Abstract: Neural networks that can grow or both grow and shrink during learning, referred to as growing neural networks and elastic neural networks, respectively, have recently been explored in offline continual learning with a particular focus on catastrophic forgetting.
Related stories
arXiv Machine Learning
Jul 28
Gradient-Free Continual Learning
arXiv:2504. 01219v2 Announce Type: replace Abstract: Neural networks are notorious for forgetting old skills when taught new ones - a problem known as catastrophic forgetting.
arXiv AI
Jul 22
Parameter-Efficient Continual Fine-Tuning: A Survey
arXiv:2504. 13822v3 Announce Type: replace-cross Abstract: The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance.
arXiv Machine Learning
Jun 9
Evaluating the Impact of Task Granularity on Catastrophic Forgetting in Continual Learning
arXiv:2606. 08013v1 Announce Type: new Abstract: Catastrophic forgetting, the abrupt loss of previously acquired knowledge upon learning new information, remains the central challenge in Continual Learning.
arXiv AI
6d ago
Representation Finetuning for Continual Learning
arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.
arXiv AI
Jun 9
Preserving Plasticity in Continual Learning via Dynamical Isometry
arXiv:2606. 09762v1 Announce Type: cross Abstract: Continual training of deep neural networks under non-stationarity often leads to a progressive loss of plasticity, eventually limiting further learning.
arXiv Machine Learning
6d ago
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
Hugging Face Trending Papers
Jun 8
Preserving Plasticity in Continual Learning via Dynamical Isometry
Continual training of deep neural networks under non-stationarity often leads to a progressive loss of plasticity, eventually limiting further learning. We relate plasticity to the empirical Neural Tangent Kernel, and identify dynamical isometry (the condition that layer-wise Jacobian singular values remain close to one) as a key mechanism for preserving plasticity in continual learning.
arXiv Machine Learning
Jul 10
When Does Continual Learning Require Learning
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
arXiv Machine Learning
Jun 9
Theoretical Foundations of Continual Learning via Drift-Plus-Penalty
arXiv:2606. 08452v1 Announce Type: new Abstract: In many real-world settings, data streams are nonstationary and arrive sequentially, requiring learning systems to adapt continuously without retraining from scratch.
arXiv Machine Learning
Jul 30
The Art of Not Forgetting A Local Learning Architecture for Continual Learning
arXiv:2607. 26523v1 Announce Type: new Abstract: We introduce CMP (Cognitive Memory Primitive), a continual-learning architecture that repre?
arXiv Machine Learning
Jun 11
On the Stability of Growth in Structural Plasticity
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.