arXiv Machine Learning
Binding-Specific Relearning Savings for Test-Time Continual Learning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Oct 12
Self-Preservation in Language Models
The paper investigates how language models can aid their own continual learning by generating replay data and selecting preservation targets. By organizing task descriptions into Task Banks, the authors show that replay generated by frozen models reduces forgetting on tasks outside the bank and that informed selection of replay targets can improve retention. The study demonstrates that models can support broader self‑preservation, especially when larger models are used, and that replacing new‑task examples with replay can enhance preservation under a fixed example budget.
arXiv Machine Learning
Jul 20
Rethinking Transfer in Continual Learning: A Replay-Based Realisation
arXiv:2607. 15587v1 Announce Type: new Abstract: Continual learning studies how deployed language models can continually acquire new tasks without expensive retraining from scratch.
arXiv Machine Learning
Jul 8
Fine-Tuning Regimes Define Distinct Continual Learning Problems
arXiv:2604. 21927v3 Announce Type: replace Abstract: Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge.
arXiv Machine Learning
4d ago
Not All Change Is Damage: Update-Matched Unlearning for Large Language Models
The paper introduces Update‑Matched Unlearning (UMU), a method that applies the same simulated relearning update to both a defended model and an undefended reference to better preserve plasticity while resisting relearning of deleted knowledge. Experiments on Llama and Qwen models with TOFU and FaithUn datasets show that UMU reduces harmful and benign relearning by 30.4 and 9.9 points on average and increases new‑task accuracy by 3.5 points, supporting the reference‑timing hypothesis. The study demonstrates that robust unlearning should selectively prevent recovery of deleted knowledge without suppressing the model’s ability to learn new tasks.
arXiv Computation and Language
Oct 8
A Deafening Silence: Catastrophic Forgetting Lives in the Output Embeddings of Tokens the Data Never Speaks
arXiv:2610.09835v1 Announce Type: new Abstract: Continual pre-training and fine-tuning in Large Language Models (LLMs) inevitably induce catastrophic forgetting, typically mitigated by replay using o...
arXiv Machine Learning
Jul 10
When Does Continual Learning Require Learning
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?