arXiv Computation and Language October 23, 2026 By Shuo Cai, Yanggan Gu, Yuhang Liu, Su Lu, Yuanyi Wang, Yang Yu, Zhen Li, Zhitian Hou, Guanghao Zhu, Ming Li, Hongxia Yang

FusionEval: Benchmarking Model Fusion for Large Language Models

Read the original on arXiv Computation and Language →

FusionEval is a new benchmark for evaluating model fusion techniques that combine multiple expert models into a single large language model. It provides controlled expert pools trained from Qwen3 and Llama on domains such as math, code, science, instruction following, and tool use, and evaluates capability retention, computational cost, runtime, and GPU memory across eight public benchmarks. Experiments show that training‑free fusion methods keep reasoning performance but hurt tool use, while MOPD maintains tool use and balances capabilities at the cost of extra data preparation and training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Jul 31

Divergence Decoding: Training-Free Capability Fusion

arXiv:2607. 27248v1 Announce Type: cross Abstract: While large language models excel in reasoning, these generalists often lack knowledge for specialized scientific domains.

By Yimi Wang, Hao Li, Shuo Yang, He Cao, Dechen Zhang, Ziang Wu, Zhiyuan Yan, Fanyang Mo, Li Yuan

llms benchmarks

arXiv Machine Learning

Jul 2

FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data

arXiv:2507. 10540v3 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has created a diverse landscape of models, each excelling at different tasks.

By Tao Feng, Haozhen Zhang, Zijie Lei, Pengrui Han, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jiaxuan You

llms fine-tuning efficiency benchmarks

arXiv Computation and Language

Aug 28

Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms

The paper investigates three fusion paradigms—Merge, Mix RL, and multi‑teacher on‑policy distillation (MOPD)—for consolidating reinforcement learning with verifiable rewards (RLVR) across multiple domains. Experiments across model scales and a multi‑domain benchmark show that while overall performance differences are small, significant gaps can appear on specific tasks, and each method exhibits distinct training dynamics and constraints. Practical guidelines are offered: Merge for cheap fusion when experts exist, Mix RL for unified training with adjustable domain mixtures, and MOPD when preserving domain‑specific gains is paramount.

By Siye Wu, Kai Yang, Yuchen Cai, Xin Xu, Peng-Yuan Wang, Jiaxuan Wang, Jiashun Liu, Jiafei Lyu, Yangkun Chen, Saiyong Yang, Yanghua Xiao

llms reinforcement-learning efficiency benchmarks

arXiv AI

Jun 2

Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills

arXiv:2503. 05641v4 Announce Type: replace-cross Abstract: Combining existing pre-trained LLMs is a promising approach for diverse reasoning tasks.

By Justin Chih-Yao Chen, Sukwon Yun, Elias Stengel-Eskin, Tianlong Chen, Mohit Bansal

llms agents benchmarks

arXiv AI

Jun 26

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

arXiv:2606. 26671v1 Announce Type: new Abstract: Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization.

By Qiaobo Hao, Yangqian Wu, Shunyi Wang, Zhongjian Zhang, Ziqun Li, Yayin He, Muqing Li, Chen Zhong

llms reinforcement-learning fine-tuning efficiency benchmarks safety

arXiv AI

Jul 21

Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective

arXiv:2607. 18026v1 Announce Type: new Abstract: Can large language models with substantially different parameter spaces be merged by direct weighted averaging, without training or semantic alignment?

By Jiahe Fan, Yinghao Hou, Si Chen, Aiyuan Zhang, Hong Xie, Defu Lian

llms efficiency benchmarks safety

One email a morning, machine-written

Related stories

Divergence Decoding: Training-Free Capability Fusion

FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data

Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms

Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective