arXiv AI

Efficient LLM Collaboration via Planning

arXiv Computation and Language
Oct 16

MoCo: A One-Stop Shop for Model Collaboration Research

MoCo is a Python library that consolidates 26 model collaboration methods for language models, enabling execution, benchmarking, and comparison at scale. It includes 25 evaluation datasets across reasoning, QA, code, safety, and more, allowing users to add their own data. Experiments show that most collaboration strategies outperform non‑collaborative models in 61.0% of settings, with top methods achieving up to 25.8% improvement.

By Shangbin Feng, Yuyang Bai, Ziyuan Yang, Yike Wang, Zhaoxuan Tan, Jiajie Yan, Zhenyu Lei, Wenxuan Ding, Weijia Shi, Haojin Wang, Zhenting Qi, Yuru Jiang, Heng Wang, Chengsong Huang, Yu Fei, Jihan Yao, Yilun Du, Luke Zettlemoyer, Yejin Choi, Yulia Tsvetkov
arXiv AI
Oct 12

Collaborative Learning and Reasoning over Model Slices

The paper introduces collaborative learning for large language models, where a single large model is replaced by smaller, heterogeneous models—called model slices—under a comparable parameter budget. By enabling prior reasoning to be reassessed across different model parameterizations and employing pyramid supervision that blends protocol distillation with step‑level supervision, the authors refine reasoning and interaction decisions during collaboration. Experiments on multi‑hop question answering, mathematical reasoning, code generation, and open‑ended constrained generation show that the trained heterogeneous models match or surpass the performance of the original large model and established multi‑agent baselines, while also achieving higher inference efficiency and exhibiting task‑specific collaboration patterns.

By Yuheng Wang, Runde Yang, Zhuohan Bao, Qiran Zhang, Huatao Li, Yiyang Zheng, Lin Wu, Jingru Fan, Shu Yao, Yihan Li, Zhi Li, Chen Qian
arXiv AI
Oct 7

DHCG: Dynamic Construction of Hierarchical Collaboration Graphs for LLM-Based Multi-Agent Reasoning

The paper introduces DHCG, a framework that dynamically constructs hierarchical collaboration graphs for large language model–based multi‑agent systems. DHCG coordinates Planner, Worker, and Generator modules to adaptively determine the composition and scale of agents during execution, guided by feedback and action‑aware preference optimization. Experiments on code generation, mathematical reasoning, and domain‑specific tasks show DHCG surpasses static and dynamic baselines, improving performance by 2.77–8.02 points and achieving a 13.06‑point gain over single‑agent baselines.

By Jie Ren, Jiakang Yuan, Chenyu Huang, Hezeer Ma, Jiayuan Fan, Tao Chen
arXiv AI
Oct 28

AgentBaton: Efficient Model Collaboration in Agentic Tasks

AgentBaton explores how capable and efficient language models can collaborate on long‑horizon agentic tasks by handoffs, where one model takes over from another. The study finds that handoffs from a capable to an efficient model at appropriate stages preserve or improve accuracy while significantly cutting cost, whereas the reverse handoff is less effective. AgentBaton, a lightweight detector, learns the optimal handoff point and achieves an average API cost reduction of 65.9% across 11 model pairs while largely maintaining task accuracy, even gaining up to 8.8 percentage points over capable‑only execution.

By Junlong Li, Jindong Jiang, Yonggan Fu, Jiarui Yao, Hanrong Ye, Wonmin Byeon, Peter Belcak, Hao Zhang, Pavlo Molchanov
arXiv Computation and Language
Nov 2

AgentCollab: A Self-Evaluation-Driven Collaboration Paradigm for Efficient LLM Agents

AgentCollab introduces a self-evaluation-driven collaboration paradigm for large language model agents, dynamically coordinating models of varying reasoning capacities during task execution. By using the agent’s own self-reflection to decide when to elevate to a stronger model, it avoids reliance on external routing modules and allocates additional reasoning budget only when recent failures indicate difficulty. Experiments on multi-step agent benchmarks demonstrate that this two-level small-large model setup consistently improves the accuracy-efficiency Pareto frontier of LLM agents.

By Wenbo Gao, Renxi Liu, Xian Wang, Fang Guo, Shuai Yang, Xi Chen, Hui-Ling Zhen, Hanting Chen, Weizhe Lin, Xiaosong Li, Yaoyuan Wang
arXiv AI
Nov 5

MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability

MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.

By Qizhi Chu, Zekai Yu, Sijie Wen, Yang Liu, Chen Qian, Cheng Yang, Chuan Shi, Zhiyuan Liu