The paper investigates three fusion paradigms—Merge, Mix RL, and multi‑teacher on‑policy distillation (MOPD)—for consolidating reinforcement learning with verifiable rewards (RLVR) across multiple domains. Experiments across model scales and a multi‑domain benchmark show that while overall performance differences are small, significant gaps can appear on specific tasks, and each method exhibits distinct training dynamics and constraints. Practical guidelines are offered: Merge for cheap fusion when experts exist, Mix RL for unified training with adjustable domain mixtures, and MOPD when preserving domain‑specific gains is paramount.
By Siye Wu, Kai Yang, Yuchen Cai, Xin Xu, Peng-Yuan Wang, Jiaxuan Wang, Jiashun Liu, Jiafei Lyu, Yangkun Chen, Saiyong Yang, Yanghua Xiao
arXiv:2606. 19164v1 Announce Type: cross Abstract: Model merging aims to enable multi-task learning by integrating the capabilities of multiple models fine-tuned from the same pre-trained checkpoint into a single model.
By Longhua Li, Lei Qi, Xin Geng, Qi Tian
arXiv:2606. 26902v1 Announce Type: new Abstract: Multi-task model merging aims to consolidate several task-specific experts into a unified model, yet static merging consistently suffers from parameter interference.
By Jinwook Jung, Taegyu Kim, Kumju Jo, Sungyong Baik
Multi-task model merging aims to consolidate several task-specific experts into a unified model, yet static merging consistently suffers from parameter interference. While dynamic merging models aim to bridge this gap, many works rely on the costly storage and loading of redundant expert components at inference.
CORAM introduces a new approach to merging fine‑tuned models by partitioning each target weight matrix into row slices and representing each slice with its singular value decomposition in the base‑model’s SVD frame. The method performs manifold averaging of task‑specific factors and applies an amplification coefficient to counteract contraction, with the coefficient’s scale estimated from update norms and its restoration strength chosen from expert update dispersion. Across multiple model families and scales, CORAM outperforms the prior OrthoMerge technique by up to 1.35 points and matches or exceeds the strongest weight‑space baselines.
CORAM (Coherent Orthogonal Rotation for Model Merging) is a new method for combining fine‑tuned models without joint training or access to original data. It partitions each target weight matrix into row slices, represents each expert slice with its singular value decomposition in the base‑model SVD frame, and merges the task‑specific factors on their corresponding manifolds. The approach includes an amplification coefficient to counteract manifold averaging contraction, spread slicing to balance highly updated rows, and a residual pathway for non‑target layers, achieving improvements over existing orthogonal merging techniques across multiple model families and scales.
By Xinyi Sui, Ziran Liu, Nam Ling, Wei Wang, Wei Jiang