arXiv Statistics ML
A Dynamical Theory of LoRA in Continual Learning
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Jun 19
Sparsity, Superposition, and Forgetting: A Mechanistic Study of Representation Retention in Continual Learning
arXiv:2606. 20431v1 Announce Type: new Abstract: Continual learning (CL) systems often forget previously acquired knowledge, yet the mechanisms driving forgetting remain hard to isolate in practice because real datasets entangle many factors.
arXiv Machine Learning
Jul 28
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
arXiv:2607. 23837v1 Announce Type: new Abstract: Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting.
arXiv Machine Learning
Jun 29
When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning
arXiv:2606. 28117v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models.
arXiv AI
Jun 30
Neural Subspace Reallocation: Continual Learning as Retrieval-Based Subspace Memory Management
arXiv:2606. 30067v1 Announce Type: cross Abstract: We introduce Neural Subspace Reallocation (NSR), which reframes continual learning as memory management over parameter subspaces.
arXiv AI
Jun 29
Energy-Structured Low-Rank Adaptation for Continual Learning
arXiv:2605. 27482v2 Announce Type: replace-cross Abstract: While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compaction and exhausting capacity for future tasks.
Hugging Face Trending Papers
Jun 29
Neural Subspace Reallocation: Continual Learning as Retrieval-Based Subspace Memory Management
We introduce Neural Subspace Reallocation (NSR), which reframes continual learning as memory management over parameter subspaces. Instead of treating Low-Rank Adaptation (LoRA) modules as disposable per-task adapters, NSR manages them as compressible, retrievable memory units on a frozen backbone through a recurring cycle: (1) compress learned LoRAs via SVD, (2) reserve them in a TaskKnowledgeBank, (3) recall related past LoRAs by embedding similarity to warm-start new or returning tasks, and (4) reallocate the active subspace accordingly, with distillation protecting prior tasks.