← Back to all news
arXiv Machine Learning By Juho Jung, Seongjae Kang, Jinyoung Han

Deconstructing Multi-Task Active Learning: The Paradox of Gradient Conflict and Orthogonal Decomposition

Read the original on arXiv Machine Learning →

The paper critiques current multi‑task active learning (MTL‑AL) methods that use linear scalarization of objectives like uncertainty, diversity, and gradient conflict, arguing that such combinations miss Pareto‑optimal trade‑offs. It introduces a new framework that separates acquisition into informativeness and exploration, decouples conflict scoring from candidate ranking, and uses gradient surgery during training. The authors also propose a cross‑task mutual information metric and demonstrate consistent improvements on COCO, Cityscapes, and NYUv2 datasets compared to recent baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 25

Transformers as Cross-Task Learners: Shared Structure Drives Sample Efficiency in In-Context Learning

Transformers can learn broad families of tasks during pretraining and adapt to unseen tasks from a short prompt, but a rigorous understanding of this capability is limited. This paper studies how shared cross‑task structure influences the sample complexity of in‑context learning (ICL) by characterizing task‑space complexity through covering numbers, yielding a set of anchor functions that localize unseen tasks and predict responses. The authors construct a Transformer with Softmax attention to approximate this procedure and derive an error bound that separates the effects of pretraining tasks and prompt length, showing that once enough tasks are available the dependence on prompt length becomes dimension‑free.

By Zhongjie Shi, Rongjie Lai, Alexander Cloninger, Wenjing Liao
llms
More like this →
arXiv Machine Learning
Jun 16

Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion

arXiv:2312. 06173v2 Announce Type: replace Abstract: Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multi-task model that performs well across diverse tasks.

By Anke Tang, Xianglin Luo, Li Shen, Yong Luo, Liang Ding, Han Hu, Bo Du, Dacheng Tao
fine-tuning
More like this →