The paper introduces Fisher-Shaped EWC (FS‑EWC), a method that extends Elastic Weight Consolidation by controlling Fisher sensitivity over a neighborhood of potential future updates. FS‑EWC minimizes the worst‑case empirical Fisher information trace during task learning, favoring solutions less sensitive to bounded future displacement, and uses trajectory‑based model merging to construct the retained solution. Experiments on multiple benchmarks show that FS‑EWC outperforms standard EWC, achieving a better stability–plasticity balance and greater tolerance to parameter drift.
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
By Tieliang Gong, Zhongbo Zhang, Wen Wen, Yong-Jin Liu
arXiv:2610.19876v1 Announce Type: new
Abstract: Continual learning aims to acquire new knowledge while retaining previously learned knowledge. However, updating shared parameters when learning new ta...
By Teif Aldaajani, Huda Hakami, Maha A. Thafar
arXiv:2607. 09202v1 Announce Type: cross Abstract: Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation.
By Julius St\"ork
The paper introduces Trust Region Continual Learning, a hybrid approach that blends generative replay with a Fisher-metric trust region constraint to address catastrophic forgetting in sequential task learning. It demonstrates that, under local approximations, the update can be interpreted as a single-step Model-Agnostic Meta-Learning (MAML) process, where replay provides an old-task gradient and the Fisher penalty shapes curvature. Experiments on task-incremental diffusion image generation and continual diffusion-policy control show that this method achieves superior final performance, better retention, and faster early-task recovery compared to EWC, replay, and other continual meta-learning baselines.
By Zekun Wang, Anant Gupta, Christopher J. MacLellan
arXiv:2606. 01379v1 Announce Type: new Abstract: While prompt-based parameter-efficient continual learning mitigates catastrophic forgetting by isolating task-specific prompts, this isolation also limits later tasks from improving earlier ones, leaving backward knowledge transfer underexplored.
By Anushka Tiwari, Kaiyi Ji
The paper introduces TMLN (Trajectory-Modulatory Landscape Navigation), a method that treats continual learning as an optimal control problem on a curved loss landscape. It uses a diagonal empirical Fisher Information Matrix to approximate a local Riemannian manifold and dynamically modulates a preconditioner based on the network’s historical parameter trajectory. This trajectory‑based preconditioning is integrated into gradient updates to protect important parameter directions without adding explicit penalties, and experiments on class‑ and domain‑incremental benchmarks show a significant reduction in the loss barrier between tasks.
By Isabelle Aguilar, Zayn Andre Zainal, Luis Fernando Herbozo Contreras, Zhaojing Huang, Omid Kavehei
arXiv:2606. 08452v1 Announce Type: new Abstract: In many real-world settings, data streams are nonstationary and arrive sequentially, requiring learning systems to adapt continuously without retraining from scratch.
By Nazreen Shah, Govinda Arya, Bharath B. N., Ranjitha Prasad
arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.
By Haihua Luo, Xuming Ran, Tommi K\"arkk\"ainen, Huiyan Xue, Zhonghua Chen, Qi Xu, Fengyu Cong
arXiv:2607. 15587v1 Announce Type: new Abstract: Continual learning studies how deployed language models can continually acquire new tasks without expensive retraining from scratch.
By Yang Meng, Zhenya Liu, Zhuokai Zhao, Yuxin Chen
arXiv:2610.18038v2 Announce Type: replace-cross
Abstract: World action models (WAMs) built on pretrained video generation models adapt quickly to new robotic tasks, but they forget previously learned...
By Kaipeng Fang, Hangxu Liu, Chaojian Shi, Yunsong Zhou, Hongzi Zhu, Minyi Guo, Jingkuan Song, Lianli Gao
The paper introduces an algorithmic stability framework to analyze catastrophic forgetting in continual learning using SGD, without assuming a specific model architecture. It decomposes forgetting into intra‑task and cross‑task generalization gaps plus empirical forgetting, and derives bounds for different learning‑rate schedules. Experiments on Split CIFAR‑100 and Split MiniImageNet illustrate the theoretical insights.
By Guodong Zheng, Yan Sun, Peng Wang, Enneng Yang, Jiafei Lyu, Xiaochun Cao, Dacheng Tao, Li Shen