← Back to all news
arXiv Machine Learning By Kathrin Korte, Christian Metzner, Akshay Bedhotiya, Viet Anh Khoa Tran, Eleni Nisioti, Sebastian Risi

Implicit Assumptions About Tasks in Continual Learning

Read the original on arXiv Machine Learning →

The paper investigates how implicit assumptions about tasks in continual learning affect reported performance metrics. It distinguishes between the underlying data, the partitioning of that data into tasks, and the learner’s experience, showing that different granularities, orders, and similarity structures can lead to markedly different outcomes. The authors use controlled teacher–student experiments to illustrate these effects and review existing benchmarks to assess which properties are explicitly stated.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 23

Fine-Grained Analysis of Catastrophic Forgetting in Continual Learning via Algorithmic Stability

The paper introduces an algorithmic stability framework to analyze catastrophic forgetting in continual learning using SGD, without assuming a specific model architecture. It decomposes forgetting into intra‑task and cross‑task generalization gaps plus empirical forgetting, and derives bounds for different learning‑rate schedules. Experiments on Split CIFAR‑100 and Split MiniImageNet illustrate the theoretical insights.

By Guodong Zheng, Yan Sun, Peng Wang, Enneng Yang, Jiafei Lyu, Xiaochun Cao, Dacheng Tao, Li Shen
safety
More like this →
arXiv Machine Learning
Sep 15

Realistic Continual Learning Approach using Pre-trained Models

arXiv:2404.07729v2 Announce Type: replace Abstract: Continual learning (CL) evaluates adaptability in learning solutions to retain knowledge. Our research addresses the challenge of catastrophic forg...

By Nadia Nasri, Carlos Guti\'errez-\'Alvarez, Sergio Lafuente-Arroyo, Saturnino Maldonado-Basc\'on, Roberto J. L\'opez-Sastre
benchmarks
More like this →
arXiv Computation and Language
Sep 22

Time-Incremental Continued Pretraining of LLMs: Knowledge Updates Without Catastrophic Forgetting

The paper investigates time‑incremental continued pretraining (CPT) of large language models using web‑scale data that overlaps across snapshots. Across six open‑weight models, CPT improves factual recall without catastrophic forgetting, while the cost is negligible and data quality outweighs quantity. The study identifies optimal learning rates, shows LoRA can match full CPT, and demonstrates that CPT gains transfer to fine‑tuned models.

By F{\i}rat \"Oncel, Salman Hussain Ali, Mirco Ravanelli, Cem Subakan, \c{C}a\u{g}atay Y{\i}ld{\i}z
llmsreinforcement-learningfine-tuningdata-engineering
More like this →