arXiv AI
Aug 20

Forgetting, plasticity, and co-observation: a third facet of continual learning

The paper argues that catastrophic forgetting and loss of plasticity alone cannot explain why naive sequential training underperforms offline joint training. It introduces data co-observation as a third factor, showing that observing training data together consistently improves performance across supervised and self-supervised settings. The study also reinterprets common continual learning methods, suggesting that memory replay’s success stems from restoring co-observation benefits rather than merely mitigating forgetting.

By Timm Hess, Abhishek Jha, Gido M. van de Ven, Tinne Tuytelaars
arXiv AI
Sep 7

MePo++: Unifying Representation Refinement and Reconciliation for General Continual Learning

MePo++ is a post‑training framework designed for general continual learning (GCL) that unifies representation refinement and reconciliation. It introduces MetaPrep, which enhances representation plasticity via unsupervised meta‑refinement on pseudo continual sequences, and StreamAlign, which maintains stability by reconciling online features with a stable pretrained geometry. Experiments across various pretrained models, datasets, and continual learning baselines show that MePo++ consistently improves performance in PTM‑based GCL.

By Guanglong Sun, Kanglei Zhou, Liyuan Wang, Qi Cheng, Hongwei Yan, Shuang Cui, Hang Su, Jun Zhu, Yi Zhong