arXiv Machine Learning October 15, 2026 By Lingfeng He, De Cheng, Huaijie Wang, Xi Yang, Nannan Wang, Xinbo Gao

Task-Driven Subspace Decomposition for Knowledge Sharing and Isolation in LoRA-based Continual Learning

Read the original on arXiv Machine Learning →

The paper introduces LoDA, a LoRA-based continual learning method that decomposes update spaces into task‑shared and truly task‑specific subspaces using energy‑based objectives. By fixing down‑projections and learning up‑projections with Gradient‑Aligned Optimization, LoDA recalibrates general updates in closed form after each task, aiming to improve knowledge sharing and isolation. Experiments show that LoDA outperforms existing CL methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

fine-tuning

arXiv AI

4d ago

Data-Aware Model Merging in the Essential Subspace for Multi-Task Knowledge Aggregation

arXiv:2606.19164v4 Announce Type: replace-cross Abstract: Model merging enables training-free multi-task learning by aggregating knowledge from models fine-tuned from a shared checkpoint, but their t...

By Longhua Li, Lei Qi, Qi Tian, Xin Geng

fine-tuning

arXiv AI

Oct 7

CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling

arXiv:2610.08312v1 Announce Type: new Abstract: Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, rec...

By Maoqi Liu, Quan Fang, Yufei He

llms fine-tuning benchmarks

arXiv AI

Jul 28

Continual Knowledge Consolidation LORA for Domain Incremental Learning

arXiv:2510. 16077v2 Announce Type: replace-cross Abstract: Domain Incremental Learning (DIL) is a sub-branch of continual learning that aims to address the never-ending arrival of new domains without catastrophic forgetting.

By Naeem Paeedeh, Mahardhika Pratama, Weiping Ding, Jimmy Cao, Wolfgang Mayer, Ryszard Kowalczyk, Ary Shiddiqi

fine-tuning benchmarks safety

arXiv Machine Learning

Sep 1

One Adapter, Many Tasks: Task-Conditioned Feature Transformations for Continual Learning

arXiv:2608.31096v1 Announce Type: cross Abstract: Class-incremental learning (CIL) requires a model to incrementally learn tasks that contain new classes without accessing earlier training data while...

By Yunxiang Fu, Meng Lou, Yizhou Yu

fine-tuning

arXiv AI

Jun 29

Energy-Structured Low-Rank Adaptation for Continual Learning

arXiv:2605. 27482v2 Announce Type: replace-cross Abstract: While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compaction and exhausting capacity for future tasks.

By Longhua Li, Lei Qi, Qi Tian, Xin Geng

diffusion fine-tuning benchmarks

arXiv Machine Learning

Aug 31

When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging

arXiv:2608.27518v1 Announce Type: new Abstract: Continual learning (CL) and model merging (MM) both aim to obtain a single model that performs well across multiple tasks, challenged respectively by c...

By Shangge Liu, Yuehan Yin, Yinghuan Shi, Lei Wang, Wenbin Li

benchmarks

One email a morning, machine-written

Related stories

Data-Aware Model Merging in the Essential Subspace for Multi-Task Knowledge Aggregation

CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling

Continual Knowledge Consolidation LORA for Domain Incremental Learning

One Adapter, Many Tasks: Task-Conditioned Feature Transformations for Continual Learning

Energy-Structured Low-Rank Adaptation for Continual Learning

When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging