arXiv Machine Learning
Jul 28
Gradient-Free Continual Learning
arXiv:2504. 01219v2 Announce Type: replace Abstract: Neural networks are notorious for forgetting old skills when taught new ones - a problem known as catastrophic forgetting.
Mistake gating leads to energy and memory efficient continual learning
arXiv:2604. 14336v2 Announce Type: replace Abstract: Synaptic plasticity is metabolically expensive, yet animals continuously update their internal models without exhausting energy reserves.
Related stories
arXiv Machine Learning
Aug 18
Metaplasticity as adaptive gradient preconditioning for incremental learning
arXiv:2608. 14634v1 Announce Type: new Abstract: Biological intelligence naturally prevents catastrophic forgetting through Complementary Learning Systems (CLS) theory, a macroscopic consolidation process driven at the local level by synaptic metaplasticity: the continuous, history-dependent neuromodulation of individual synapses.
arXiv AI
Jun 24
Fast and Slow Variational Continual Learning
arXiv:2606. 24007v1 Announce Type: cross Abstract: Continual learning remains a major challenge for modern deep networks, partly because commonly used optimizers lack inherent mechanisms for continual adaptation.
arXiv AI
Jul 22
Parameter-Efficient Continual Fine-Tuning: A Survey
arXiv:2504. 13822v3 Announce Type: replace-cross Abstract: The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance.
arXiv Machine Learning
Aug 4
Plasticity of Growing and Elastic Neural Networks in Online Continual Learning
arXiv:2608. 01475v1 Announce Type: new Abstract: Neural networks that can grow or both grow and shrink during learning, referred to as growing neural networks and elastic neural networks, respectively, have recently been explored in offline continual learning with a particular focus on catastrophic forgetting.
arXiv AI
Jun 9
Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks
arXiv:2606. 08447v1 Announce Type: cross Abstract: One of the critical limitations of artificial neural networks is their lack of ability to continually learn: training on new tasks often leads to interference and forgetting of the previous ones.
arXiv Machine Learning
Aug 13
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
arXiv Machine Learning
Jul 10
When Does Continual Learning Require Learning
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
arXiv Machine Learning
2d ago
ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning
arXiv:2609.38833v1 Announce Type: new Abstract: Federated continual learning must integrate new tasks over time without losing earlier-task knowledge. Most existing methods attach an anti-forgetting...
arXiv AI
Aug 6
NeuMoSync: End-to-End Neuromodulatory Control for Plasticity and Adaptability in Continual Learning
arXiv:2608. 04358v1 Announce Type: new Abstract: Continual learning (CL) requires models to learn tasks sequentially, yet deep neural networks often suffer from plasticity loss and poor knowledge transfer, which can impede their long-term adaptability.
arXiv Machine Learning
Jun 9
Theoretical Foundations of Continual Learning via Drift-Plus-Penalty
arXiv:2606. 08452v1 Announce Type: new Abstract: In many real-world settings, data streams are nonstationary and arrive sequentially, requiring learning systems to adapt continuously without retraining from scratch.
arXiv AI
Aug 21
In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models
arXiv:2608. 19514v1 Announce Type: cross Abstract: Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems.