The paper introduces Fisher-Shaped EWC (FS‑EWC), a method that extends Elastic Weight Consolidation by controlling Fisher sensitivity over a neighborhood of potential future updates. FS‑EWC minimizes the worst‑case empirical Fisher information trace during task learning, favoring solutions less sensitive to bounded future displacement, and uses trajectory‑based model merging to construct the retained solution. Experiments on multiple benchmarks show that FS‑EWC outperforms standard EWC, achieving a better stability–plasticity balance and greater tolerance to parameter drift.
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
By Tieliang Gong, Zhongbo Zhang, Wen Wen, Yong-Jin Liu
arXiv:2610.19876v1 Announce Type: new
Abstract: Continual learning aims to acquire new knowledge while retaining previously learned knowledge. However, updating shared parameters when learning new ta...
By Teif Aldaajani, Huda Hakami, Maha A. Thafar
arXiv:2607. 09202v1 Announce Type: cross Abstract: Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation.
By Julius St\"ork
The paper introduces Trust Region Continual Learning, a hybrid approach that blends generative replay with a Fisher-metric trust region constraint to address catastrophic forgetting in sequential task learning. It demonstrates that, under local approximations, the update can be interpreted as a single-step Model-Agnostic Meta-Learning (MAML) process, where replay provides an old-task gradient and the Fisher penalty shapes curvature. Experiments on task-incremental diffusion image generation and continual diffusion-policy control show that this method achieves superior final performance, better retention, and faster early-task recovery compared to EWC, replay, and other continual meta-learning baselines.
By Zekun Wang, Anant Gupta, Christopher J. MacLellan
arXiv:2606. 01379v1 Announce Type: new Abstract: While prompt-based parameter-efficient continual learning mitigates catastrophic forgetting by isolating task-specific prompts, this isolation also limits later tasks from improving earlier ones, leaving backward knowledge transfer underexplored.
By Anushka Tiwari, Kaiyi Ji