The paper investigates how coordination among AI agents serving different users degrades performance compared to a single coordinating agent. Across five advanced models and 77 scenarios in four shared-resource environments—API key budgets, clinic calendars, personal assistant bookings, and merge queues—the study finds that multi‑agent teams consistently underperform, sometimes collapsing entirely, and that even with communication channels coordination overhead remains significant. The authors identify specific failure modes such as stalling, action overriding, and claim fabrication, and propose environment‑specific mitigations like team leads and procedural instructions, while releasing the MAMUBench benchmark for future research.
By Sahan Paliskara, Nattaput Namchittai, Andrew Lampinen
arXiv:2607. 27155v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly expected to assist users in completing tasks.
By Jingbo Zhou, Yusai Zhao, Qi Bao, Jingjia Cao, Zhenghai Chen, Chang Gao, Kaiqi Guo, Muxin Guo, Mingxuan Li, Xinjiang Lu, Yanru Ma, Yixiong Xiao, Zenghui Zhang, Le Zhang, Hua Wu
arXiv:2512. 11213v2 Announce Type: replace Abstract: Scaling test-time computation has been shown to significantly improve large language model (LLM) performance without additional training.
By Dongwon Jung, Peng Shi, Muhao Chen, Yi Zhang
arXiv:2511. 02734v3 Announce Type: replace Abstract: Current evaluations of Large Language Model (LLM) agents primarily emphasize task completion, often overlooking resource efficiency and adaptability.
By Jiayu Liu, Cheng Qian, Zhaochen Su, Qing Zong, Shijue Huang, Bingxiang He, Yi R. Fung
Consilience is an inference‑time orchestration framework that steers and certifies communication among multi‑agent large language models in hidden‑profile settings. It summarizes each discussion turn with a compact state of uncertainty, disagreement, evidence gain, redundancy, and premature consensus, then selects a communication intervention (challenge, clarify, seek evidence, or route) and speaker. A round‑wise conformal calibration procedure guarantees that the controller’s proposed action has bounded one‑step regret with high probability, and an acceptance mechanism enforces this guarantee for the executed action. Experiments on HiddenBench‑style tasks show that Consilience improves decision accuracy and communication efficiency over fixed and unstructured protocols, sometimes outperforming a full‑information baseline.
By Abhijith Babu, Ramneet Kaur, Vishal Pramanik, Olivera Kotevska, Nathaniel D. Bastian, Susmit Jha, Sunny Raj, Yanzhao Wu, Sumit Kumar Jha, Anirban Roy
arXiv:2606. 28061v1 Announce Type: cross Abstract: Large language models (LLMs) have increasingly moved from standalone text generation systems to agents that invoke external tools, access environments, and execute multi-step tasks.
By Shijing Hu, Liang Liu, Zhu Meng, Zhicheng Zhao
Large language model (LLM) agents are increasingly expected to assist users in completing tasks. However, existing benchmarks provide limited support for evaluating whether agents can carry out office-suite workflows at a reasonable cost.
arXiv:2512. 16310v3 Announce Type: replace-cross Abstract: LLM-based agents increasingly use multiple external tools to complete complex tasks.
By Yuxuan Qiao, Dongqin Liu, Hongchang Yang, Wei Zhou, Songlin Hu
arXiv:2606.23189v2 Announce Type: replace-cross
Abstract: Computer-use agents (CUAs) now act on a user's behalf across personal applications such as email, calendars, and to-do lists. This cross-appl...
By Anmol Goel, Iryna Gurevych
DuMateBench is a new benchmark for autonomous agents that uses real user sessions from a large production platform, preserving interaction history, configurations, and workspace state. It contains 200 tasks across 8 scenarios and 17 capability categories, many requiring coordination of multiple capabilities. The benchmark tests agents in Docker containers with real-world complexities—Insufficient, Unstable, and Noisy—and evaluates performance with a hybrid deterministic and LLM-as-Judge protocol, revealing significant gaps in task completion across various agent frameworks and LLMs.
By Zechun Niu, Yukun Zhao, Jiaxin Zhang, Xu Shen, Jinhua Si, Han Tian, Can Xu, Yunfan Song, Jiaxin Mao, Yansong Gao, Yuchen Li, Jianmin Wu, Lingyong Yan, Shuaiqiang Wang, Dawei Yin
arXiv:2606. 10662v1 Announce Type: cross Abstract: Multi-agent systems (MAS) can scale large language model reasoning at test time by decomposing complex problems into parallel subtasks.
By Yuzhen Mao, Azalia Mirhoseini
arXiv:2602. 11510v3 Announce Type: replace Abstract: Multi-agent Large Language Model (LLM) systems create privacy risks that current output-only benchmarks cannot measure.
By Faouzi El Yagoubi, Godwin Badu-Marfo, Ranwa Al Mallah