arXiv AI

STABLE: A Continual Learning Optimizer with Adaptive Drift Control

STABLE is a primal‑dual optimizer that controls parameter drift to prevent catastrophic forgetting in continual learning. By enforcing a drift budget through adaptive dual variables, it guarantees bounded margin erosion for distance‑based classifiers under a local score‑stability condition. Experiments on unimodal and multimodal streams show that models trained with STABLE match or outperform specialized continual learning methods while preserving zero‑shot transfer.

arXiv Machine Learning
Oct 28

Trust Region Continual Learning as an Implicit Meta-Learner

The paper introduces Trust Region Continual Learning, a hybrid approach that blends generative replay with a Fisher-metric trust region constraint to address catastrophic forgetting in sequential task learning. It demonstrates that, under local approximations, the update can be interpreted as a single-step Model-Agnostic Meta-Learning (MAML) process, where replay provides an old-task gradient and the Fisher penalty shapes curvature. Experiments on task-incremental diffusion image generation and continual diffusion-policy control show that this method achieves superior final performance, better retention, and faster early-task recovery compared to EWC, replay, and other continual meta-learning baselines.

By Zekun Wang, Anant Gupta, Christopher J. MacLellan
arXiv Machine Learning
Oct 23

Fine-Grained Analysis of Catastrophic Forgetting in Continual Learning via Algorithmic Stability

The paper introduces an algorithmic stability framework to analyze catastrophic forgetting in continual learning using SGD, without assuming a specific model architecture. It decomposes forgetting into intra‑task and cross‑task generalization gaps plus empirical forgetting, and derives bounds for different learning‑rate schedules. Experiments on Split CIFAR‑100 and Split MiniImageNet illustrate the theoretical insights.

By Guodong Zheng, Yan Sun, Peng Wang, Enneng Yang, Jiafei Lyu, Xiaochun Cao, Dacheng Tao, Li Shen