arXiv AI
6d ago
Representation Finetuning for Continual Learning
arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.
When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning
arXiv:2606. 28117v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models.
Related stories
arXiv Machine Learning
Jul 28
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
arXiv:2607. 23837v1 Announce Type: new Abstract: Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting.
arXiv Machine Learning
Aug 12
GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging
arXiv:2509. 13211v4 Announce Type: replace Abstract: The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models.
arXiv AI
Jul 20
PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
arXiv:2601. 13020v2 Announce Type: replace-cross Abstract: Continual instruction tuning (CIT) requires multimodal large language models (MLLMs) to adapt to a stream of tasks without forgetting prior capabilities.
arXiv Machine Learning
Jun 29
TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree
arXiv:2506. 10355v2 Announce Type: replace Abstract: Many real-world applications collect data in a streaming environment, where learning tasks are encountered sequentially.
arXiv Machine Learning
Jun 2
Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning
arXiv:2603. 11653v2 Announce Type: replace Abstract: Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments.
arXiv AI
Jun 29
Energy-Structured Low-Rank Adaptation for Continual Learning
arXiv:2605. 27482v2 Announce Type: replace-cross Abstract: While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compaction and exhausting capacity for future tasks.
arXiv AI
Jul 10
ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning
arXiv:2607. 07719v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning adapts a large language model to one task cheaply, but across a task sequence LoRA-style methods keep stacking low-rank updates on the same frozen weight, so each new task tends to overwrite the previous ones.
arXiv AI
Jul 22
Parameter-Efficient Continual Fine-Tuning: A Survey
arXiv:2504. 13822v3 Announce Type: replace-cross Abstract: The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance.
arXiv Machine Learning
Jul 8
Fine-Tuning Regimes Define Distinct Continual Learning Problems
arXiv:2604. 21927v3 Announce Type: replace Abstract: Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge.
arXiv Machine Learning
Jul 3
Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation
arXiv:2602. 00722v2 Announce Type: replace Abstract: Parameter-efficient continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge.
arXiv AI
Jul 28
Continual Knowledge Consolidation LORA for Domain Incremental Learning
arXiv:2510. 16077v2 Announce Type: replace-cross Abstract: Domain Incremental Learning (DIL) is a sub-branch of continual learning that aims to address the never-ending arrival of new domains without catastrophic forgetting.