arXiv:2506.11578v5 Announce Type: replace
Abstract: Recently, large language models (LLMs) have demonstrated strong performance, ranging from simple to complex tasks. However, while large models achi...
By Byeongchan Lee, Jonghoon Lee, Dongyoung Kim, Jaehyung Kim, Kyungjoon Park, Dongjun Lee, Jinwoo Shin
The paper introduces collaborative learning for large language models, where a single large model is replaced by smaller, heterogeneous models—called model slices—under a comparable parameter budget. By enabling prior reasoning to be reassessed across different model parameterizations and employing pyramid supervision that blends protocol distillation with step‑level supervision, the authors refine reasoning and interaction decisions during collaboration. Experiments on multi‑hop question answering, mathematical reasoning, code generation, and open‑ended constrained generation show that the trained heterogeneous models match or surpass the performance of the original large model and established multi‑agent baselines, while also achieving higher inference efficiency and exhibiting task‑specific collaboration patterns.
By Yuheng Wang, Runde Yang, Zhuohan Bao, Qiran Zhang, Huatao Li, Yiyang Zheng, Lin Wu, Jingru Fan, Shu Yao, Yihan Li, Zhi Li, Chen Qian
arXiv:2609.38816v1 Announce Type: new
Abstract: While multi-agent and model collaboration algorithms gain traction to combine the strengths of diverse Large Language Models (LLMs), existing systems r...
By Zongwan Cao, Ziyuan Yang, Shangbin Feng, Michael Duan, Skyler Hallinan, Bingbing Wen, Lucy Lu Wang, Yulia Tsvetkov
While multi-agent and model collaboration algorithms gain traction to combine the strengths of diverse Large Language Models (LLMs), existing systems remain bottlenecked on pre-defined and hand-crafte...
COMED (Controlled Model Escalation for Multi-LLM Deliberation) is a post-anchor controller that selectively engages cross-model collaboration in multi-LLM inference. It uses anchor self‑consistency, router margin, and a lightweight peer probe to accept confident answers, verify ambiguous cases, and only escalates when collaboration is likely beneficial. Experiments on medical, scientific, and general reasoning benchmarks show that COMED improves performance across 16 open‑weight settings, achieving up to +10.7 percentage points on MedQA and outperforming dense collaboration while invoking fewer models and decoded tokens.
By Norah Alballa, Wenxuan Zhang, Salma Kharrat, Fares Fourati, Zafar Ayyub Qazi, Mohamed Elhoseiny, Marco Canini
arXiv:2608. 08605v1 Announce Type: new Abstract: Multi-agent systems (MAS) built on Large Language Models (LLMs) are proliferating rapidly, but their heterogeneous execution traces provide no common basis for evaluation across methods.
By Guo Chen, Ziwen Li, Reed Li, Yu Lu, Haibo Shi, Bingbing Xu, Junjie Huang