The paper introduces Fisher-Shaped EWC (FS‑EWC), a replay‑free continual learning method that extends Elastic Weight Consolidation by controlling Fisher sensitivity over a neighborhood of future updates. FS‑EWC minimizes the worst‑case empirical Fisher information trace during task learning, favoring solutions robust to bounded parameter displacement, and uses trajectory‑based model merging to construct the retained solution from multiple optimization states. Experiments on several benchmarks demonstrate that FS‑EWC outperforms standard EWC, achieving a better stability–plasticity balance and higher tolerance to parameter drift.
By Liying Wang, Simone Magistri, Joost van de Weijer
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
By Tieliang Gong, Zhongbo Zhang, Wen Wen, Yong-Jin Liu
arXiv:2606. 01379v1 Announce Type: new Abstract: While prompt-based parameter-efficient continual learning mitigates catastrophic forgetting by isolating task-specific prompts, this isolation also limits later tasks from improving earlier ones, leaving backward knowledge transfer underexplored.
By Anushka Tiwari, Kaiyi Ji
arXiv:2606. 08452v1 Announce Type: new Abstract: In many real-world settings, data streams are nonstationary and arrive sequentially, requiring learning systems to adapt continuously without retraining from scratch.
By Nazreen Shah, Govinda Arya, Bharath B. N., Ranjitha Prasad
The paper introduces Trust Region Continual Learning, a hybrid approach that blends generative replay with a Fisher-metric trust region constraint to address catastrophic forgetting in sequential task learning. It demonstrates that, under local approximations, the update can be interpreted as a single-step Model-Agnostic Meta-Learning (MAML) process, where replay provides an old-task gradient and the Fisher penalty shapes curvature. Experiments on task-incremental diffusion image generation and continual diffusion-policy control show that this method achieves superior final performance, better retention, and faster early-task recovery compared to EWC, replay, and other continual meta-learning baselines.
By Zekun Wang, Anant Gupta, Christopher J. MacLellan
arXiv:2610.19876v1 Announce Type: new
Abstract: Continual learning aims to acquire new knowledge while retaining previously learned knowledge. However, updating shared parameters when learning new ta...
By Teif Aldaajani, Huda Hakami, Maha A. Thafar
arXiv:2607. 09202v1 Announce Type: cross Abstract: Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation.
By Julius St\"ork
arXiv:2607. 15587v1 Announce Type: new Abstract: Continual learning studies how deployed language models can continually acquire new tasks without expensive retraining from scratch.
By Yang Meng, Zhenya Liu, Zhuokai Zhao, Yuxin Chen
arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.
By Haihua Luo, Xuming Ran, Tommi K\"arkk\"ainen, Huiyan Xue, Zhonghua Chen, Qi Xu, Fengyu Cong
The paper introduces TMLN (Trajectory-Modulatory Landscape Navigation), a method that treats continual learning as an optimal control problem on a curved loss landscape. It uses a diagonal empirical Fisher Information Matrix to approximate a local Riemannian manifold and dynamically modulates a preconditioner based on the network’s historical parameter trajectory. This trajectory‑based preconditioning is integrated into gradient updates to protect important parameter directions without adding explicit penalties, and experiments on class‑ and domain‑incremental benchmarks show a significant reduction in the loss barrier between tasks.
By Isabelle Aguilar, Zayn Andre Zainal, Luis Fernando Herbozo Contreras, Zhaojing Huang, Omid Kavehei
arXiv:2607. 26523v1 Announce Type: new Abstract: We introduce CMP (Cognitive Memory Primitive), a continual-learning architecture that repre?
By Ashmith Atmuri, Yashaswini Rao Bhogarajula
We introduce CMP (Cognitive Memory Primitive), a continual-learning architecture that repre? sents inputs as sparse relational codes, stores them in a two-tier competitive memory, and learns through local updates without end-to-end backpropagation through its feature-generating system.