arXiv AI
Jul 8
To Retain or to Adapt? Generalizing Continual Learning
arXiv:2607. 05609v1 Announce Type: cross Abstract: The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting.
PMF-CL: Pareto-Minimal-Forgetting Continual Learner for Conflicting Tasks
arXiv:2605. 19145v3 Announce Type: replace Abstract: In the literature, many continual learning (CL) algorithms have been proposed to address the issue of catastrophic forgetting in ML models (i.
Related stories
arXiv AI
6d ago
Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models
The paper introduces EoupCT, a framework that estimates and orthogonalizes unknown pre‑training gradients to mitigate catastrophic forgetting during continual fine‑tuning of large language models. It generates pseudo data most susceptible to forgetting using a learnable soft prompt with Gumbel‑Softmax, then jointly optimizes model parameters and the prompt via a first‑order Pareto optimizer to enforce orthogonality between new task updates and the estimated gradients. Experiments on multiple LLMs show that EoupCT preserves both task‑specific performance and the models’ inherent general‑purpose knowledge.
arXiv Machine Learning
Aug 31
When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging
arXiv:2608.27518v1 Announce Type: new Abstract: Continual learning (CL) and model merging (MM) both aim to obtain a single model that performs well across multiple tasks, challenged respectively by c...
arXiv Machine Learning
Jun 30
The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning
arXiv:2606. 29832v1 Announce Type: new Abstract: Machine unlearning aims to eliminate the influence of specific data from trained models to safeguard privacy.
arXiv AI
2d ago
From Order to Distribution: An Exact Operator Framework for Forgetting in Continual Learning
arXiv:2604.13460v2 Announce Type: replace-cross Abstract: A central challenge in continual learning is forgetting: the loss of performance on previously learned tasks after learning new ones. Prior t...
arXiv Machine Learning
Jun 29
Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
arXiv:2510. 18874v3 Announce Type: replace Abstract: Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities -- a phenomenon classically known as catastrophic forgetting.
arXiv Machine Learning
Aug 28
Unifying Detection and Adaptation in Task-Free Continual Learning
The paper introduces FiUni, a Fisher-guided unified framework that performs batch-level task detection and parameter-efficient continual adaptation for large language models. By exploiting orthogonality in the Fisher information matrix’s Kronecker-Factored Approximate Curvature subspaces, FiUni constructs frozen subspaces to guide low-rank adaptation and matches incoming batch windows to historical subspaces. This approach allows the model to decide whether to reuse, expand, or create new subspaces, balancing knowledge sharing and task isolation while achieving competitive performance with fewer trainable parameters.
Hugging Face Trending Papers
Jun 29
The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning
Machine unlearning aims to eliminate the influence of specific data from trained models to safeguard privacy. However, this presents a significant challenge in the context of continual learning (CL), where models update sequentially on dynamic datasets.
arXiv AI
Jun 8
Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learning
arXiv:2505. 12239v2 Announce Type: replace-cross Abstract: In Continual Learning (CL), using a Pre-Trained Model (PTM) as the feature extractor has become a popular practice.
arXiv Machine Learning
2d ago
ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning
arXiv:2609.38833v1 Announce Type: new Abstract: Federated continual learning must integrate new tasks over time without losing earlier-task knowledge. Most existing methods attach an anti-forgetting...
arXiv Machine Learning
Jun 9
Theoretical Foundations of Continual Learning via Drift-Plus-Penalty
arXiv:2606. 08452v1 Announce Type: new Abstract: In many real-world settings, data streams are nonstationary and arrive sequentially, requiring learning systems to adapt continuously without retraining from scratch.
arXiv Machine Learning
2d ago
SeqLoRA: Bilevel Orthogonal Adaptation for Continual Multi-Concept Generation
arXiv:2605.22743v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning enables fast personalization of text-to-image diffusion models to user-provided concepts (objects, people, or style...