The paper introduces In-Context Preference-Conditioned Pareto Set Learning (ICPSL), a framework that uses a pretrained hypernetwork to generate a problem-specific, preference-conditioned Pareto solution model from limited offline data. ICPSL eliminates the need for repeated preference-wise searches or problem-specific training by producing a reusable model in a single forward pass. Experiments on synthetic and real-world problems demonstrate that ICPSL yields high-quality Pareto solutions while significantly speeding up model construction.
By Yiming Yao, Liang Zhao, Qingfu Zhang
arXiv:2606. 15115v1 Announce Type: new Abstract: Multi-objective optimization (MOO) has emerged as a powerful approach to solving complex optimization problems involving multiple objectives.
By Yiyi Zhu, Yaolin Wen, Xiang Xia, Xin An, Hanyi Si, Xiang Shu, Yangde Fu, Liang Dou, Hong Qian
arXiv:2606. 19521v1 Announce Type: new Abstract: In multi-task learning, handling an increasing number of objectives can quickly become challenging, both in terms of the computational resources and the decision maker's capacity to choose appropriate trade-offs.
By Augustina C. Amakor, Konstantin Sonntag, Sebastian Peitz
Spectral-MoE introduces a method for allocating heterogeneous per-layer expert utilization in dense-to-mixture-of-experts (MoE) conversion of large language models. It offers both static and dynamic allocators that learn to distribute a utilization budget across layers, improving performance over uniform allocation in both analytic- and learning-based conversion baselines. Experiments on Llama-2-7B, Llama-3-8B, and Qwen2.5-7B show significant reductions in perplexity and accuracy gains at various FFN utilizations.
By Zachary Broveak, Christian Kubicka, Thierry Tambe
arXiv:2605. 01712v2 Announce Type: replace Abstract: Pareto set learning (PSL) is an emerging paradigm in multi-objective optimization that trains neural networks to map preference vectors to Pareto optimal solutions.
By Xinyue Chen, Yingxuan Liang, Yiqin Huang, Chikai Shang, Hai-Lin Liu, Fangqing Gu
arXiv:2606. 01666v1 Announce Type: cross Abstract: The scaling of Large Language Models (LLMs) has driven significant performance gains but created substantial challenges in inference efficiency.
By Udbhav Bamba, Arnav Chavan, Aryamaan Thakur, Steve Teig, Deepak Gupta