The Flow has not summarised this story yet — read it at arXiv Computation and Language.
MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.
The paper examines when multi‑agent collaboration is beneficial versus single‑agent approaches. It finds that collaboration yields systematic advantages mainly in long‑horizon tasks with sparse dependencies, while single agents perform better in tightly coupled, sequential workflows. The authors introduce SAIGE, a lightweight multi‑agent mechanism that models collaboration as a dynamically evolving graph, and show that it balances context efficiency and task performance without always improving outcomes as more agents are added.
The paper examines when multi‑agent collaboration is truly beneficial as large language models grow more capable. It finds that multi‑agent systems yield systematic advantages mainly for long‑horizon tasks with sparse dependencies, while single‑agent approaches excel in tightly coupled, sequential workflows. The authors introduce SAIGE, a lightweight, graph‑based collaboration framework that balances context efficiency and performance, demonstrating that adding more agents or deeper recursion does not always improve outcomes.
The paper investigates how large language model agents handle long‑horizon tasks and finds that verification is crucial for success but often occurs too late. It shows that sharing a common context dilutes duty‑relevant information and that simply separating duties among agents is insufficient, as agents can lose sight of the overall goal. To address these issues, the authors propose Goal‑Aware Dynamic Multi‑Agent Collaboration (GAD), which uses an execution graph to organize subtasks, keeps each agent’s context focused on its specific goal, and dynamically adjusts task dependencies and information sharing based on execution evidence, with verification and recovery checks providing feedback for repair.
arXiv:2512. 11213v2 Announce Type: replace Abstract: Scaling test-time computation has been shown to significantly improve large language model (LLM) performance without additional training.
The paper introduces Goal-Aware Dynamic Multi-Agent Collaboration (GAD), a framework that assigns distinct duties to separate agents with individualized contexts to tackle long-horizon tasks. It employs a growing execution graph to structure subtasks and their dependencies, ensuring each agent retains the original instruction, task goal, and acceptance conditions. GAD dynamically adapts collaboration and information sharing, using verification and recovery checks to monitor subtask results and overall task completion, providing feedback for repair when checks fail.