arXiv Computation and Language

CONCAT: Consensus- and Confidence-Driven Ad Hoc Teaming for Efficient LLM-Based Multi-Agent Systems

CONCAT is a training‑free framework that improves the efficiency of large language model (LLM) based multi‑agent systems by clustering agents according to their initial answers and selecting cluster leaders based on confidence. It uses a Theory‑of‑Mind‑inspired heuristic to predict collaboration benefits between leaders, then prunes communications to form an ad‑hoc network that reduces latency. Experiments on three LLMs and benchmarks show up to 2.02× higher accuracy/latency ratio than LLM‑Debate and a 50.1% latency reduction on Qwen2.5‑14B‑Instruct without task‑specific training.

arXiv AI
Jun 17

Algorithmic Prompt Generation for Diverse Human-like Teaming and Communication with Large Language Models

arXiv:2504. 03991v2 Announce Type: replace-cross Abstract: Understanding how humans collaborate and communicate in teams is essential for improving human-agent teaming and AI-assisted decision-making.

By Siddharth Srikanth, Varun Bhatt, Boshen Zhang, Werner Hager, Charles Michael Lewis, Katia P. Sycara, Aaquib Tabrez, Stefanos Nikolaidis
arXiv AI
Oct 1

CollabFlow: Recursive Self-Improvement of Agent Collaboration

CollabFlow introduces a recursive self‑improvement framework for multi‑agent collaboration in large language model systems. It trains a Collab‑Director to assemble teams of agents, uses a frozen executor to run them, and retrains the director each round based on outcomes. The system incorporates evidence‑conditioned communication protocols within collaboration graphs and a Collaborative Trajectory Balance objective to maintain diverse high‑performing teams across rounds, achieving superior performance on twelve datasets.

By Xiao Huang, Mingda Zhang, Junming Zhang, Qiang Huang, Hanwen Zhang, Yue Dai, Zijia Wang, Xiaoying Tang
arXiv Computation and Language
Oct 1

You're Hired: Strategic Model Selection for LLM Collaboration

arXiv:2609.38816v1 Announce Type: new Abstract: While multi-agent and model collaboration algorithms gain traction to combine the strengths of diverse Large Language Models (LLMs), existing systems r...

By Zongwan Cao, Ziyuan Yang, Shangbin Feng, Michael Duan, Skyler Hallinan, Bingbing Wen, Lucy Lu Wang, Yulia Tsvetkov
arXiv AI
Aug 24

Consilience: Conformally Calibrated Communication Control for Hidden-Profile Multi-Agent Reasoning

Consilience is an inference‑time orchestration framework that steers and certifies communication among multi‑agent large language models in hidden‑profile settings. It summarizes each discussion turn with a compact state of uncertainty, disagreement, evidence gain, redundancy, and premature consensus, then selects a communication intervention (challenge, clarify, seek evidence, or route) and speaker. A round‑wise conformal calibration procedure guarantees that the controller’s proposed action has bounded one‑step regret with high probability, and an acceptance mechanism enforces this guarantee for the executed action. Experiments on HiddenBench‑style tasks show that Consilience improves decision accuracy and communication efficiency over fixed and unstructured protocols, sometimes outperforming a full‑information baseline.

By Abhijith Babu, Ramneet Kaur, Vishal Pramanik, Olivera Kotevska, Nathaniel D. Bastian, Susmit Jha, Sunny Raj, Yanzhao Wu, Sumit Kumar Jha, Anirban Roy
arXiv Machine Learning
Oct 15

Narrowing the Collaboration Gap, Probably

arXiv:2610.19215v1 Announce Type: new Abstract: Large language models are increasingly deployed as teams of agents that hold different private information about a shared task. How should such agents...

By Mirah Shi, Marcel Hussing, Ira Globus-Harris, Natalie Collina, Aaron Roth, Surbhi Goel
arXiv AI
Jun 9

Benchmarking Open-Ended Multi-Agent Coordination in Language Agents

arXiv:2606. 08340v1 Announce Type: new Abstract: As language models are increasingly deployed as autonomous agents, they must coordinate with others over long horizons in open-ended interactive tasks.

By Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker, Alexander Rutherford, Davide Paglieri, Aidan Scannell, Henry Gouk, Elliot J. Crowley, Tim Rockt\"aschel, Amos Storkey
arXiv AI
Jul 7

Optimal-Agent-Selection: State-Aware Routing Framework for Efficient Multi-Agent Collaboration

arXiv:2511. 02200v2 Announce Type: replace Abstract: The emergence of multi-agent systems powered by large language models (LLMs) has unlocked new frontiers in complex task-solving, enabling diverse agents to integrate unique expertise, collaborate flexibly, and address challenges unattainable for individual models.

By Jingbo Wang, Sendong Zhao, Haochun Wang, Yuzheng Fan, Ting Liu
arXiv AI
Oct 12

Competence-Aware Orchestration for LLM-Based Multi-Agent Systems

The paper introduces a two-stage competence-aware framework for orchestrating large language model (LLM)-based multi-agent systems (MAS). It first estimates the LLM backbone’s task-conditioned competence for a given query, then uses this estimate to adjust which roles are activated and how agents communicate. Across six benchmarks, the method improves average performance to 93.81% and, depending on backbone competence, can boost accuracy while reducing token costs.

By Songmiao Wang, Yuchen Wang, Chenglin Yu, Hongxia Yang, Ming Li
arXiv AI
Nov 5

MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability

MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.

By Qizhi Chu, Zekai Yu, Sijie Wen, Yang Liu, Chen Qian, Cheng Yang, Chuan Shi, Zhiyuan Liu
arXiv AI
Jul 13

Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks

arXiv:2607. 09330v1 Announce Type: new Abstract: Embodied agent teams powered by heterogeneous large language models (LLMs) are being widely deployed in physical artificial intelligence such as smart factories, warehouses, and service robotics.

By Nuocheng Yang, Sihua Wang, Zihan Chen, Tony Q. S. Quek, Changchuan Yin