arXiv AI

Zero-Shot Coordination among LLM Agents

The paper investigates zero‑shot coordination (ZSC) among large language model (LLM) agents, a scenario where independently developed agents must coordinate without prior shared scaffolding. Experiments in minimal, controlled environments reveal that current frontier LLMs struggle to coordinate, mainly due to limited understanding of the coordination problem and weak reasoning about partners’ beliefs. The authors introduce Coordination‑Friendly Definitions (CFDs) as a principled, automatically discoverable approach to improve robust coordination among LLM agents.

arXiv AI
Jun 9

Benchmarking Open-Ended Multi-Agent Coordination in Language Agents

arXiv:2606. 08340v1 Announce Type: new Abstract: As language models are increasingly deployed as autonomous agents, they must coordinate with others over long horizons in open-ended interactive tasks.

By Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker, Alexander Rutherford, Davide Paglieri, Aidan Scannell, Henry Gouk, Elliot J. Crowley, Tim Rockt\"aschel, Amos Storkey
arXiv AI
Oct 26

Symbolic Guidance for LLM Agents in Distributed Multiagent Coordination

The paper introduces the Symbolic Guidance Taxonomy (SGT) to regulate the autonomy of large language model agents in distributed multi‑agent coordination. By providing symbolic guidance—ranging from open‑ended reasoning to fully prescriptive algorithmic steps—the authors find that intermediate levels of autonomy consistently outperform both unguided agents and fully prescriptive specifications. These results suggest that carefully controlling agent autonomy is a crucial design principle for effective LLM‑based coordination.

By Ben Rachmut, Ning Zhang, Yevgeniy Vorobeychik, William Yeoh
arXiv AI
Jul 29

Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm

arXiv:2607. 25446v1 Announce Type: new Abstract: Multi-agent frameworks built on large language models (LLMs) routinely entangle three logically distinct concerns: who is on the team (organization), how members align (coordination), and which algorithm fuses their work (collaboration protocol).

By Huan Chen, Xiang Song, Jian Jin, Pan Ren, Liang-Jie Zhang
arXiv AI
3d ago

STRIATUM-CTF: A Protocol-Driven Agentic Framework for Autonomous CTF Solving

STRIATUM-CTF is a modular agentic framework that uses the Model Context Protocol to mediate between large language model reasoning and the execution of offensive‑security tools. By exposing tools as strictly typed, schema‑validated functions, the protocol rejects invalid tool calls and returns structured observations that the agent can use to self‑correct. In controlled experiments the system achieved 77.8–86.7% success across 15 CTF challenges, and in a live multi‑team CTF at Florida State University it placed first among more than 30 teams.

By James Hugglestone, Samuel Jacob Chacko, Dawson Stoller, Ryan Schmidt, Xiuwen Liu
arXiv AI
1d ago

MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability

MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.

By Qizhi Chu, Zekai Yu, Sijie Wen, Yang Liu, Chen Qian, Cheng Yang, Chuan Shi, Zhiyuan Liu
arXiv AI
Jul 14

AgentAbstain: Do LLM Agents Know When Not to Act?

arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.

By Xun Liu, Yi Evie Zhang, Vira Kasprova, Parisa Rabbani, Pardis Sadat Zahraei, Tianyu Zhang, Ali Ebrahimpour-Boroojeny, Varun Chandrasekaran
arXiv AI
Aug 12

CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification

arXiv:2604. 01687v3 Announce Type: replace Abstract: Anthropic proposes the concept of skills for LLM agents to tackle multi-step professional tasks that simple tool invocations cannot address.

By Hanrong Zhang (Steve), Shicheng Fan (Steve), Henry Peng Zou (Steve), Yankai Chen (Steve), Zhenting Wang (Steve), Jiayu Zhou (Steve), Chengze Li (Steve), Wei-Chieh Huang (Steve), Yifei Yao (Steve), Kening Zheng (Steve), Xue (Steve), Liu, Xiaoxiao Li, Philip S. Yu
arXiv AI
6d ago

Goal-Aware Dynamic Multi-Agent Collaboration for Long-Horizon Tasks

The paper introduces Goal-Aware Dynamic Multi-Agent Collaboration (GAD), a framework that assigns distinct duties to separate agents with individualized contexts to tackle long-horizon tasks. It employs a growing execution graph to structure subtasks and their dependencies, ensuring each agent retains the original instruction, task goal, and acceptance conditions. GAD dynamically adapts collaboration and information sharing, using verification and recovery checks to monitor subtask results and overall task completion, providing feedback for repair when checks fail.

By Xu Shen, Lingyong Yan, Shuaiqiang Wang, Jianmin Wu, Zhikai Chen, Jiliang Tang, Jie Ren