The paper introduces Iterative Sequential Transfer (IST), a method for few-shot multiobjective multitask optimization that addresses the challenge of aligning elite solution distributions across tasks. IST treats multitask optimization as a sequence of transfer problems, focusing evaluations on one target task per iteration and using a likelihood-informed prioritization to select the task most ready for knowledge integration. Experiments on benchmark and real-world problems demonstrate the method’s effectiveness under tight evaluation budgets.
By Tingyang Wei, Haofeng Wu, Ananda Phan Iman, Zhao Wei, Jiao Liu, Yew-Soon Ong
The paper presents Iterative Sequential Transfer (IST), a method for few-shot multiobjective multitask optimization that addresses the bottleneck of aligning elite solution distributions across tasks. IST treats multitask optimization as a sequence of transfer problems, focusing evaluations on one target task per iteration and using a likelihood-informed prioritization to select the task most ready for knowledge integration. Experiments on benchmark and real-world problems demonstrate IST’s effectiveness under tight evaluation budgets.
arXiv:2406. 09770v2 Announce Type: replace-cross Abstract: Solving multi-objective optimization problems for large deep neural networks is a challenging task due to the complexity of the loss landscape and the expensive computational cost of training and evaluating models.
By Anke Tang, Li Shen, Yong Luo, Shiwei Liu, Han Hu, Bo Du, Dacheng Tao
arXiv:2606. 15115v1 Announce Type: new Abstract: Multi-objective optimization (MOO) has emerged as a powerful approach to solving complex optimization problems involving multiple objectives.
By Yiyi Zhu, Yaolin Wen, Xiang Xia, Xin An, Hanyi Si, Xiang Shu, Yangde Fu, Liang Dou, Hong Qian
SIMS: Scale-Invariant Merit-Function-Based Scalarization for Multi-Task Learning proposes a new scalarization method for multi-task learning that is invariant to the relative scales of task losses. By using a logarithmic transformation, SIMS converts the multi-objective problem into a single objective that preserves weak Pareto optimality and allows a smooth surrogate with controllable approximation error. Experiments on standard multi-task benchmarks show that SIMS consistently outperforms existing scalarization methods and achieves state‑of‑the‑art performance.
By Zebin Chen, Fei Xing, Yang Chen, Hua Liu, Andy HF Chow, Yuhua Qian, Yu Zhang
arXiv:2606. 19521v1 Announce Type: new Abstract: In multi-task learning, handling an increasing number of objectives can quickly become challenging, both in terms of the computational resources and the decision maker's capacity to choose appropriate trade-offs.
By Augustina C. Amakor, Konstantin Sonntag, Sebastian Peitz
arXiv:2608. 04926v1 Announce Type: cross Abstract: As chart images, tabular data, and visualization code play increasingly important roles across diverse domains, cross-representation understanding across these modalities poses fundamental challenges for AI systems: the relationships across representations are inherently \textit{one-to-many}, supervision is ambiguous and costly, and model optimization lacks a principled signal that is both direction-adaptive and representation-generalizable beyond task-specific objectives.
By Xuehang Guo, Pengyuan Li, Tom Hope, Tirthankar Ghosal, Manling Li, Qingyun Wang
Transformers can learn broad families of tasks during pretraining and adapt to unseen tasks from a short prompt, but a rigorous understanding of this capability is limited. This paper studies how shared cross‑task structure influences the sample complexity of in‑context learning (ICL) by characterizing task‑space complexity through covering numbers, yielding a set of anchor functions that localize unseen tasks and predict responses. The authors construct a Transformer with Softmax attention to approximate this procedure and derive an error bound that separates the effects of pretraining tasks and prompt length, showing that once enough tasks are available the dependence on prompt length becomes dimension‑free.
By Zhongjie Shi, Rongjie Lai, Alexander Cloninger, Wenjing Liao
arXiv:2606. 02221v1 Announce Type: cross Abstract: Multi-task learning (MTL) aims to construct a joint model for multiple tasks by sharing a common representation across domains.
By Chengfeng Wu, Tao Zou, Yanru Wu, Jingge Wang
arXiv:2607. 27953v1 Announce Type: new Abstract: Combinatorial optimization problems (COPs) underpin many real-world decisions, but their exponentially large search spaces make high-quality solutions costly to obtain.
By Shengda Gu, Kai Li, Xinyi Ke, Haobo Fu, Yifan Zhang, Jian Cheng
arXiv:2607. 03522v1 Announce Type: new Abstract: Fine-tuning a single low-rank adapter on many domains at once is multi-task learning: the domains must be co-learned, and how they share the adapter decides whether they help or hurt one another.
By Wei Zhang, Lin Tang, Ming Zhao, Yuxuan Wang
arXiv:2509. 16959v5 Announce Type: replace-cross Abstract: When different objectives conflict with each other in multi-task learning, gradients begin to interfere and slow convergence, thereby potentially reducing the final model's performance.
By Santosh Patapati, Ian Noronha