arXiv:2411. 16073v4 Announce Type: replace-cross Abstract: Inspired by the Well-initialized Lottery Ticket Hypothesis (WLTH), we introduce Soft-TransFormers (Soft-TF), a continual learning framework that adapts a frozen pre-trained Transformer through task-specific soft subnetworks: real-valued multiplicative masks over the query, key, value, and output projections of selected self-attention layers.
By Haeyong Kang, Chang D. Yoo
arXiv:2604. 27031v2 Announce Type: replace-cross Abstract: In a continual learning setting, we require a model to be plastic enough to learn a new task and stable enough to not disturb previously learned capabilities.
By Karthik Charan Raghunathan, Christian Metzner, Laura Kriener, Melika Payvand
arXiv:2607. 09202v1 Announce Type: cross Abstract: Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation.
By Julius St\"ork
arXiv:2606. 30067v1 Announce Type: cross Abstract: We introduce Neural Subspace Reallocation (NSR), which reframes continual learning as memory management over parameter subspaces.
By Byeong Hoon Yoon
We introduce Neural Subspace Reallocation (NSR), which reframes continual learning as memory management over parameter subspaces. Instead of treating Low-Rank Adaptation (LoRA) modules as disposable per-task adapters, NSR manages them as compressible, retrievable memory units on a frozen backbone through a recurring cycle: (1) compress learned LoRAs via SVD, (2) reserve them in a TaskKnowledgeBank, (3) recall related past LoRAs by embedding similarity to warm-start new or returning tasks, and (4) reallocate the active subspace accordingly, with distillation protecting prior tasks.
FedSEAM is a federated continual learning framework that trains large language models across multiple clients without sharing data, using sparse mixture-of-experts to adapt selected parameters on a frozen backbone. It introduces task-aware tile selection for new tasks and a subspace-based compatibility test to decide which expert tiles to reuse or replace. On the server, Subspace Expert Agreement Merging (SEAM) aligns client updates only along agreed subspaces, while additional protections mitigate routing drift and merge erasure, preserving prior knowledge during training.
By Fatema Siddika, Abhijit Chunduru, Zilinghan Li, Ali Jannesari, Ravi Madduri
arXiv:2602. 03846v2 Announce Type: replace-cross Abstract: We develop a continual learning method for pretrained models that \emph{requires no access to old-task data}, addressing a practical barrier in foundation model adaptation where pretraining distributions are often unavailable.
By Romain Cosentino
ENCL is a neuromorphic continual learning method that uses a lightweight local learning rule and a sub‑network freezing strategy to train new tasks efficiently. By avoiding memory‑intensive experience replay and costly global backpropagation, ENCL reduces memory usage, training time, power, and energy consumption while maintaining competitive accuracy on task‑incremental learning benchmarks.
By Rachmad Vidya Wicaksana Putra, Shuochen Zhou, Hassan Raza, Muhammad Shafique
arXiv:2608.22215v1 Announce Type: new
Abstract: Large language model (LLM) agents operate in dynamic environments where knowledge continuously evolves. Existing memory systems typically treat externa...
By Wenzhi Li, Dong Nie, Rui Lan, Tongtong Lyu, Peiyao Wang, Lingzi Hong, Weihang Pan, Boyuan Pan, Yao Hu
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
By Tieliang Gong, Zhongbo Zhang, Wen Wen, Yong-Jin Liu
arXiv:2607. 07847v1 Announce Type: new Abstract: As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn?
By Anne Harrington, Nayan Saxena, Michael Murphy, Anastasia Borovykh, Zeyu Yun, Sridhar Kamath, Ara Eindra Kyi, Trevor Darrell, Jitendra Malik, Yutong Bai
arXiv:2608. 01252v1 Announce Type: new Abstract: Catastrophic forgetting is a major problem in task-incremental learning, where neural networks tend to overwrite previously learned knowledge when trained on new tasks.
By Pengxiang Wang, Hongbo Bo, Jun Hong, Weiru Liu, Kedian Mu