← Back to all news
arXiv Computer Vision By Ahmed Agiza, Tasneem Shaffee, Rahil Badkul, Sherief Reda

CRLoMTL: Conflict-Resolution through Low-rank matrices for Multi-Task Learning

Read the original on arXiv Computer Vision →

CRLoMTL is a parameter‑efficient multi‑task learning framework that mitigates task interference by operating in a low‑rank parameter‑update space rather than gradient space. It learns task‑specific LoRA updates and fuses them through a shared‑subspace resolver based on singular value decomposition, producing a single deployment‑time update that can be folded into the backbone. Experiments on Swin Transformer backbones across PASCAL‑Context, NYUD‑v2, and CelebA show that CRLoMTL improves accuracy over single‑task baselines while training only a small fraction of the parameters.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Machine Learning
Oct 15

Task-Driven Subspace Decomposition for Knowledge Sharing and Isolation in LoRA-based Continual Learning

The paper introduces LoDA, a LoRA-based continual learning method that decomposes update spaces into task‑shared and truly task‑specific subspaces using energy‑based objectives. By fixing down‑projections and learning up‑projections with Gradient‑Aligned Optimization, LoDA recalibrates general updates in closed form after each task, aiming to improve knowledge sharing and isolation. Experiments show that LoDA outperforms existing CL methods.

By Lingfeng He, De Cheng, Huaijie Wang, Xi Yang, Nannan Wang, Xinbo Gao
fine-tuning
More like this →
arXiv Machine Learning
Jun 16

Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion

arXiv:2312. 06173v2 Announce Type: replace Abstract: Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multi-task model that performs well across diverse tasks.

By Anke Tang, Xianglin Luo, Li Shen, Yong Luo, Liang Ding, Han Hu, Bo Du, Dacheng Tao
fine-tuning
More like this →