arXiv AI By Hugo Monz\'on Maldonado, Nico Daheim, Thomas M\"ollenhoff, Iryna Gurevych, Mohammad Emtiyaz Khan

Variational Model Merging for Pareto Front Estimation in Multitask Finetuning

Read the original on arXiv AI →

arXiv:2412. 08147v2 Announce Type: replace-cross Abstract: Pareto fronts are useful to find good task-mixing strategies for multitask finetuning, but they are also costly to compute.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning
Jul 30

Temporally Centered SIGReg Improves Multi-Task LeWorldModel Learning: From Analysis to Method

arXiv:2607. 26924v1 Announce Type: new Abstract: Recent work on LeWorldModel (LeWM) has shown that the Sketched Isotropic Gaussian Regularizer (SIGReg) enables stable end-to-end world-model learning from pixels by regularizing the latent marginal distribution toward an isotropic Gaussian, thereby preventing representation collapse.

By Chang Liu, Fei Suo, Yanzhou Jin, Yusuke Iwasawa, Yutaka Matsuo, Yaonan Zhu
Hugging Face Trending Papers
Jul 29

Temporally Centered SIGReg Improves Multi-Task LeWorldModel Learning: From Analysis to Method

Recent work on LeWorldModel (LeWM) has shown that the Sketched Isotropic Gaussian Regularizer (SIGReg) enables stable end-to-end world-model learning from pixels by regularizing the latent marginal distribution toward an isotropic Gaussian, thereby preventing representation collapse. While effective and elegant in single-task settings, this recipe does not extend reliably to multi-task training, leading to substantially worse downstream behavior-cloning performance.