arXiv AI
Aug 13
Representation Finetuning for Continual Learning
arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.
Forgetting, plasticity, and co-observation: a third facet of continual learning
The paper argues that catastrophic forgetting and loss of plasticity alone cannot explain why naive sequential training underperforms offline joint training. It introduces data co-observation as a third factor, showing that observing training data together consistently improves performance across supervised and self-supervised settings. The study also reinterprets common continual learning methods, suggesting that memory replay’s success stems from restoring co-observation benefits rather than merely mitigating forgetting.
Related stories
Hugging Face Trending Papers
Oct 5
Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning
The paper demonstrates that off‑policy merging, called grafting, outperforms on‑policy self‑distillation for continual learning. Grafting learns updates on an earlier donor checkpoint, scales the update, and optionally masks sensitive directions, thereby reducing interference with existing capabilities. Across various continual learning scenarios, grafting achieves better new‑task and old‑task performance without costly on‑policy sampling.
arXiv AI
Jun 3
Re-Evaluating Continual Learning with Few-Shot Adaptation
arXiv:2606. 03843v1 Announce Type: cross Abstract: Continual learning methods aim to maximize the stability and plasticity of machine learning models that are trained on a sequence of tasks.
arXiv AI
Sep 7
MePo++: Unifying Representation Refinement and Reconciliation for General Continual Learning
MePo++ is a post‑training framework designed for general continual learning (GCL) that unifies representation refinement and reconciliation. It introduces MetaPrep, which enhances representation plasticity via unsupervised meta‑refinement on pseudo continual sequences, and StreamAlign, which maintains stability by reconciling online features with a stable pretrained geometry. Experiments across various pretrained models, datasets, and continual learning baselines show that MePo++ consistently improves performance in PTM‑based GCL.
arXiv Machine Learning
Oct 8
Continual Learning without Continual Training
arXiv:2610.10379v1 Announce Type: new Abstract: Continual learning requires models to adapt to new domains and new classes while retaining prior knowledge. Many existing methods rely on continued opt...
arXiv Machine Learning
Aug 18
Geometry of Forgetting: Representation Flux in Continual Learning
arXiv:2608. 15854v1 Announce Type: new Abstract: Catastrophic forgetting remains a fundamental obstacle to continual learning, where neural networks lose previously acquired knowledge while learning new tasks.
arXiv Machine Learning
Jul 10
Are Current Continual Learning Methods Truly Agnostic? Introducing OPRE, a Step Toward Agnostic Continual Learning
arXiv:2511. 08226v2 Announce Type: replace Abstract: In order to achieve Continual Learning (CL), the problem of catastrophic forgetting, one that has plagued neural networks since their inception, must be overcome.
arXiv Machine Learning
Sep 16
CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework
arXiv:2609.17026v1 Announce Type: new Abstract: Continual learning must balance the learning of new knowledge with the retention of previously learned knowledge to incrementally learn tasks from a da...
arXiv AI
Sep 1
On the Plasticity Collapse in Continual Machine Unlearning
The paper investigates continual machine unlearning, where models must forget data over time. It identifies a fundamental issue called plasticity collapse, where successive unlearning requests cause geometric constraints that saturate parameter space, leading to two failure modes: forward failure (reduced forgetting quality) and backward failure (re‑memorization). Experiments across architectures and datasets confirm that plasticity collapse is a pervasive problem in continual unlearning.
arXiv AI
Jul 13
Interference and Retention in Continual Learning
arXiv:2607. 09202v1 Announce Type: cross Abstract: Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation.
arXiv Machine Learning
Aug 13
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
arXiv Machine Learning
Jul 10
When Does Continual Learning Require Learning
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?