arXiv:2511. 04666v4 Announce Type: replace Abstract: A fundamental challenge in developing general learning algorithms is their tendency to forget past knowledge as they adapt to new data.
arXiv:2507. 08922v3 Announce Type: replace-cross Abstract: Continual learning is an online paradigm where a learner continually accumulates knowledge from different tasks encountered over sequential time steps.
arXiv:2510. 18874v3 Announce Type: replace Abstract: Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities -- a phenomenon classically known as catastrophic forgetting.
arXiv:2610.02126v1 Announce Type: cross Abstract: We explore catastrophic forgetting in the context of large pre-trained models. By considering forgetting as a geometric problem in the input space of...
The paper argues that catastrophic forgetting and loss of plasticity alone cannot explain why naive sequential training underperforms offline joint training. It introduces data co-observation as a third factor, showing that observing training data together consistently improves performance across supervised and self-supervised settings. The study also reinterprets common continual learning methods, suggesting that memory replay’s success stems from restoring co-observation benefits rather than merely mitigating forgetting.
arXiv:2609.05966v1 Announce Type: new Abstract: Existing unlearning approaches typically rely on post hoc weight adaptation or distillation, leading to duplicated memory costs, degraded generalizatio...
arXiv:2409. 02228v2 Announce Type: replace Abstract: When language models (LMs) are trained to forget (or "unlearn'') a skill, how precisely does their behavior change?
arXiv:2607. 29503v1 Announce Type: new Abstract: While neural networks are typically evaluated by their training and test performance, these metrics do not reveal how robust a learned representation is.
arXiv:2404.07729v2 Announce Type: replace Abstract: Continual learning (CL) evaluates adaptability in learning solutions to retain knowledge. Our research addresses the challenge of catastrophic forg...
arXiv:2607. 05609v1 Announce Type: cross Abstract: The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting.
Reference-Guided Machine Unlearning (ReGUn) is a vision unlearning framework that prioritizes distributional indistinguishability over degradation-based heuristics. It uses disjoint held-out data to create a class-conditioned reference distribution for distillation, guiding forget samples toward non-member behavior without explicitly degrading predictions. Experiments across various architectures and datasets show that ReGUn achieves a competitive forgetting–utility trade-off and closely matches retrain-like membership inference behavior.
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
arXiv:2609.06986v1 Announce Type: new Abstract: Language models may need to internalize information that arrives over time and retain it through many subsequent updates. To study this challenge, we i...