arXiv Machine Learning
Continual Learning Mechanisms Compose for Long-Horizon Memorization
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 18
Task-Anchored Representation Shaping for Pre-Trained Model-Based Continual Learning
arXiv:2608. 16345v1 Announce Type: new Abstract: Pre-trained models (PTMs) provide a strong foundation for continual learning by offering stable representations that facilitate lightweight adaptation to new tasks.
arXiv Machine Learning
Jul 14
Attribution-Guided Continual Learning for Large Language Models
arXiv:2605. 05285v2 Announce Type: replace Abstract: Large language models (LLMs) often suffer from catastrophic forgetting in continual learning: after learning new tasks sequentially, they perform worse on earlier tasks.
arXiv Machine Learning
Jul 10
When Does Continual Learning Require Learning
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
arXiv AI
Oct 16
Functional Subnetwork Localization for Efficient Continual Learning in Large Language Models
The paper identifies functionally specialized subnetworks in large language models using gradient‑attribution profiling, revealing that these subnetworks have minimal overlap. It introduces CiMEO, a continual learning framework that routes each training example to its corresponding subnetwork, limiting updates to selected parameters and optimizer states. On 1.4 million sequential instructions across five generative benchmarks, CiMEO outperforms other methods while reducing catastrophic forgetting and preserving pre‑trained representations.
arXiv AI
Oct 12
Self-Preservation in Language Models
The paper investigates how language models can aid their own continual learning by generating replay data and selecting preservation targets. By organizing task descriptions into Task Banks, the authors show that replay generated by frozen models reduces forgetting on tasks outside the bank and that informed selection of replay targets can improve retention. The study demonstrates that models can support broader self‑preservation, especially when larger models are used, and that replacing new‑task examples with replay can enhance preservation under a fixed example budget.
arXiv Machine Learning
Jul 20
Rethinking Transfer in Continual Learning: A Replay-Based Realisation
arXiv:2607. 15587v1 Announce Type: new Abstract: Continual learning studies how deployed language models can continually acquire new tasks without expensive retraining from scratch.