MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.
arXiv:2611.01728v1 Announce Type: cross Abstract: Large language model (LLM)-based multi-agent systems (MAS) tackle challenging tasks by dividing work among agents and coordinating their contribution...
arXiv:2606. 04202v1 Announce Type: new Abstract: As LLMs become more widely deployed, they are increasingly expected to work alongside other AI agents rather than operating in isolation.
OpenMAS-GCom is a diagnostic benchmark designed to isolate the impact of communication structures, role assignments, and information flows in graph‑enhanced multi‑agent systems (G‑MAS). It evaluates systems by systematically modifying one component—such as rewiring communication edges, removing specialist or critic agents, or corrupting intermediate messages—while keeping tasks, models, prompts, and budget limits constant. The benchmark tests 17 configurations across 29 datasets in six domains, including 400 new G‑MAS‑Complex tasks that require agents to combine and reconcile information from multiple documents.
arXiv:2606. 10662v1 Announce Type: cross Abstract: Multi-agent systems (MAS) can scale large language model reasoning at test time by decomposing complex problems into parallel subtasks.
arXiv:2505. 11765v5 Announce Type: replace-cross Abstract: Agents powered by advanced large language models (LLMs) have demonstrated impressive capabilities across diverse complex applications.
MAS-Guard is a unified framework designed to safeguard both single- and multi-agent large language model (LLM) systems. It introduces a local Agent Guardrail (A-Guard) layer that uses task-specific experts to evaluate agentic events, and a system-level Control Plane that applies deterministic structural checks and semantic assessments to routes, capabilities, and message content. The framework enforces interventions—allow, reflection, confirmation, and block—across benchmarks, reducing attack success rates by 71.5‑86.3% while maintaining utility on benign tasks.
arXiv:2606. 28182v1 Announce Type: cross Abstract: Embodied agents operating in decentralized and partially observable environments have attracted growing attention in recent years.
arXiv:2607. 06157v1 Announce Type: cross Abstract: Deliberation plays a crucial role in collaboration; when humans work together, they naturally engage in communication to align information and reach an agreement.
arXiv:2512. 11213v2 Announce Type: replace Abstract: Scaling test-time computation has been shown to significantly improve large language model (LLM) performance without additional training.
The paper argues that the architecture of multi‑agent large language model (LLM) frameworks, rather than just the intelligence of the underlying models, largely determines system performance. It introduces a taxonomy of architectural dimensions—such as orchestration, memory, planning interfaces, specialization, and communication topology—and presents MAFBench, a unified evaluation suite. An empirical study across nine frameworks, keeping the LLM constant, reveals six design principles and shows that choices like orchestration and communication topology can dramatically affect latency, accuracy, and coordination success.
arXiv:2610.08155v1 Announce Type: cross Abstract: Large language model (LLM)-based multi-agent systems (MAS) have become a promising paradigm for complex information-seeking and reasoning tasks by en...
arXiv:2606. 05304v1 Announce Type: new Abstract: Multi-agent systems (MAS) built on large language models are typically organized around roles, pipelines, and turn schedules, while the content that agents pass to one another is often left as unconstrained natural language.