The paper introduces LoDA, a LoRA-based continual learning method that decomposes update spaces into task‑shared and truly task‑specific subspaces using energy‑based objectives. By fixing down‑projections and learning up‑projections with Gradient‑Aligned Optimization, LoDA recalibrates general updates in closed form after each task, aiming to improve knowledge sharing and isolation. Experiments show that LoDA outperforms existing CL methods.
arXiv:2606.19164v4 Announce Type: replace-cross Abstract: Model merging enables training-free multi-task learning by aggregating knowledge from models fine-tuned from a shared checkpoint, but their t...
arXiv:2610.08312v1 Announce Type: new Abstract: Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, rec...
arXiv:2510. 16077v2 Announce Type: replace-cross Abstract: Domain Incremental Learning (DIL) is a sub-branch of continual learning that aims to address the never-ending arrival of new domains without catastrophic forgetting.
arXiv:2608.31096v1 Announce Type: cross Abstract: Class-incremental learning (CIL) requires a model to incrementally learn tasks that contain new classes without accessing earlier training data while...
arXiv:2605. 27482v2 Announce Type: replace-cross Abstract: While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compaction and exhausting capacity for future tasks.
arXiv:2608.27518v1 Announce Type: new Abstract: Continual learning (CL) and model merging (MM) both aim to obtain a single model that performs well across multiple tasks, challenged respectively by c...
arXiv:2312. 06173v2 Announce Type: replace Abstract: Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multi-task model that performs well across diverse tasks.
arXiv:2606. 19164v1 Announce Type: cross Abstract: Model merging aims to enable multi-task learning by integrating the capabilities of multiple models fine-tuned from the same pre-trained checkpoint into a single model.
Geo-LoRA introduces a geometry‑aware framework for low‑rank adaptation in rehearsal‑free class‑incremental learning. It regulates the evolution of shared and task‑specific LoRA subspaces using Subspace Projection Preservation, Adaptive Core‑Slack Alignment, and Median‑Calibrated Block Overlap, ensuring smooth trajectories on the Grassmann manifold and balanced stability‑plasticity trade‑offs. The method achieves state‑of‑the‑art performance across multiple benchmarks without adding new adapter types.
arXiv:2507. 09471v4 Announce Type: replace Abstract: Continual Learning (CL) empowers AI models to continuously learn from sequential task streams.
arXiv:2606. 07500v1 Announce Type: cross Abstract: Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge.
arXiv:2606. 28117v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models.