arXiv:2507. 10540v3 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has created a diverse landscape of models, each excelling at different tasks.
By Tao Feng, Haozhen Zhang, Zijie Lei, Pengrui Han, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jiaxuan You
FusionEval is a new benchmark for evaluating model fusion techniques that combine multiple expert models into a single large language model. It provides controlled expert pools trained from Qwen3 and Llama on domains such as math, code, science, instruction following, and tool use, and evaluates capability retention, computational cost, runtime, and GPU memory across eight public benchmarks. Experiments show that training‑free fusion methods keep reasoning performance but hurt tool use, while MOPD maintains tool use and balances capabilities at the cost of extra data preparation and training.
By Shuo Cai, Yanggan Gu, Yuhang Liu, Su Lu, Yuanyi Wang, Yang Yu, Zhen Li, Zhitian Hou, Guanghao Zhu, Ming Li, Hongxia Yang
arXiv:2607. 18026v1 Announce Type: new Abstract: Can large language models with substantially different parameter spaces be merged by direct weighted averaging, without training or semantic alignment?
By Jiahe Fan, Yinghao Hou, Si Chen, Aiyuan Zhang, Hong Xie, Defu Lian
InfiFPO is a new preference optimization technique for implicit model fusion that replaces the reference model in Direct Preference Optimization with a fused source model, preserving probability information and avoiding complex vocabulary alignment. By incorporating probability clipping and max‑margin fusion, it aligns a pivot model with human preferences while distilling knowledge from multiple source models. Experiments on 11 benchmarks show that InfiFPO consistently outperforms existing fusion and preference optimization methods, raising Phi‑4’s average score from 79.95 to 83.33 and improving performance in mathematics, coding, and reasoning tasks.
By Yanggan Gu, Yuanyi Wang, Zhaoyi Yan, Yiming Zhang, Qi Zhou, Fei Wu, Hongxia Yang
arXiv:2607. 03461v1 Announce Type: cross Abstract: World models aim to capture environment dynamics in ways that support perception, reasoning, and action, and have recently become a central direction in Vision-Language-Action-World (VLAW) modeling.
By Zelin Zhao, Min Shi, Bo Yuan, Haotian Xue, Jialuo Li, Lama Moukheiber, Humphrey Shi, Yongxin Chen
arXiv:2603.13966v3 Announce Type: replace
Abstract: Vision-Language-Action (VLA) models are increasingly evaluated across multiple simulation benchmarks, yet adding each benchmark to an evaluation pi...
By Suhwan Choi, Yunsung Lee, Yubeen Park, Chris Dongjoo Kim, Ranjay Krishna, Dieter Fox, Youngjae Yu