arXiv Machine Learning

Narrowing the Collaboration Gap, Probably

arXiv AI
Aug 24

Consilience: Conformally Calibrated Communication Control for Hidden-Profile Multi-Agent Reasoning

Consilience is an inference‑time orchestration framework that steers and certifies communication among multi‑agent large language models in hidden‑profile settings. It summarizes each discussion turn with a compact state of uncertainty, disagreement, evidence gain, redundancy, and premature consensus, then selects a communication intervention (challenge, clarify, seek evidence, or route) and speaker. A round‑wise conformal calibration procedure guarantees that the controller’s proposed action has bounded one‑step regret with high probability, and an acceptance mechanism enforces this guarantee for the executed action. Experiments on HiddenBench‑style tasks show that Consilience improves decision accuracy and communication efficiency over fixed and unstructured protocols, sometimes outperforming a full‑information baseline.

By Abhijith Babu, Ramneet Kaur, Vishal Pramanik, Olivera Kotevska, Nathaniel D. Bastian, Susmit Jha, Sunny Raj, Yanzhao Wu, Sumit Kumar Jha, Anirban Roy
arXiv AI
Sep 18

Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry

The paper investigates how Large Language Model (LLM) agents can collaborate on a shared task under information asymmetry, using a table‑top version of Einstein Puzzles. It introduces a fine‑tuning‑plus‑verifier framework that equips agents with communication strategies and environmental verification signals. Results show that aligned communication is crucial for rule understanding and human trust, while a verifier improves task comprehension and promotes safer, interpretable collaboration.

By Run Peng, Ziqiao Ma, Amy Pang, Sikai Li, Zhang Xi-Jia, Yingzhuo Yu, Cristian-Paul Bara, Joyce Chai
arXiv Computation and Language
Sep 23

CONCAT: Consensus- and Confidence-Driven Ad Hoc Teaming for Efficient LLM-Based Multi-Agent Systems

CONCAT is a training‑free framework that improves the efficiency of large language model (LLM) based multi‑agent systems by clustering agents according to their initial answers and selecting cluster leaders based on confidence. It uses a Theory‑of‑Mind‑inspired heuristic to predict collaboration benefits between leaders, then prunes communications to form an ad‑hoc network that reduces latency. Experiments on three LLMs and benchmarks show up to 2.02× higher accuracy/latency ratio than LLM‑Debate and a 50.1% latency reduction on Qwen2.5‑14B‑Instruct without task‑specific training.

By Ziyang Ma, Dingyi Zhang, Sichu Liang, Jiajia Chu, Pengfei Xia, Hui Zang, Deyu Zhou
arXiv AI
Oct 1

CollabFlow: Recursive Self-Improvement of Agent Collaboration

CollabFlow introduces a recursive self‑improvement framework for multi‑agent collaboration in large language model systems. It trains a Collab‑Director to assemble teams of agents, uses a frozen executor to run them, and retrains the director each round based on outcomes. The system incorporates evidence‑conditioned communication protocols within collaboration graphs and a Collaborative Trajectory Balance objective to maintain diverse high‑performing teams across rounds, achieving superior performance on twelve datasets.

By Xiao Huang, Mingda Zhang, Junming Zhang, Qiang Huang, Hanwen Zhang, Yue Dai, Zijia Wang, Xiaoying Tang