The paper introduces Software World, a framework that simulates multi‑agent collaboration within realistic software ecosystems. It generates environments of interdependent Python packages and lets agents maintain and improve them, measuring collective speedup in hidden dependent packages. Experiments across three worlds and eight models show that performance improvement scales with inference‑time compute, with the best model boosting downstream efficiency by up to 31%.
arXiv:2606. 00953v1 Announce Type: new Abstract: Multi-agent Large Language Model (LLM) systems offer a way to decompose complex tasks, such as coding, through parallelization and context isolation.
arXiv:2603. 21489v2 Announce Type: replace-cross Abstract: AI agents have become increasingly capable at isolated software engineering (SWE) tasks such as resolving issues on Github.
MASBench is a benchmark for evaluating large language model (LLM)-based multi‑agent systems (MAS) under partial observability, a realistic constraint where agents only see limited environmental information. The benchmark is structured into three progressive task categories—Reasoning, Scheduling, and Game—and assesses three collaboration mechanisms: Protocol, Memory, and Routing. It provides deterministic evaluation metrics such as performance score, communication cost, and cost effectiveness, and includes empirical results across various LLM backbones to guide effective MAS design.
OpenCollab is a multi‑agent coding framework that unifies organization design, enforces experimental control on a shared runtime, and tracks execution via fine‑grained event streams. It introduces the metric Adherence to measure whether the declared organization is actually realized, showing that small configuration changes can shift adherence from 47.2% to 97.2%. Experiments demonstrate that a two‑coder workflow built on OpenCollab achieves new state‑of‑the‑art performance against mainstream harnesses while using the fewest tokens, and that a well‑designed organization can outperform strong existing harnesses.
arXiv:2608. 00101v1 Announce Type: cross Abstract: AI coding agents like GitHub Copilot, Claude Code, and Codex interleave multi-step LLM inference with tool execution, creating a workload different from chatbots.
arXiv:2607. 25656v1 Announce Type: new Abstract: Complex tasks often decompose into parallelizable yet interdependent subtasks, making orchestration critical to the performance of multi-agent systems (MAS).
arXiv:2603. 16011v3 Announce Type: replace-cross Abstract: Large language model (LLM) coding agents increasingly operate at the repository level, motivating benchmarks that evaluate their ability to optimize entire codebases under realistic constraints.
arXiv:2604. 04226v2 Announce Type: replace-cross Abstract: The Agentic Web is emerging as a paradigm in which autonomous software agents interact with online resources and with each other to accomplish user goals.
AgentBaton explores how capable and efficient language models can collaborate on long‑horizon agentic tasks by handoffs, where one model takes over from another. The study finds that handoffs from a capable to an efficient model at appropriate stages preserve or improve accuracy while significantly cutting cost, whereas the reverse handoff is less effective. AgentBaton, a lightweight detector, learns the optimal handoff point and achieves an average API cost reduction of 65.9% across 11 model pairs while largely maintaining task accuracy, even gaining up to 8.8 percentage points over capable‑only execution.
arXiv:2512. 11213v2 Announce Type: replace Abstract: Scaling test-time computation has been shown to significantly improve large language model (LLM) performance without additional training.
arXiv:2606. 10662v1 Announce Type: cross Abstract: Multi-agent systems (MAS) can scale large language model reasoning at test time by decomposing complex problems into parallel subtasks.
The paper "Can AI Scientists Coordinate at Runtime?" introduces Runtime Agent Coordination (RAC), a system that dynamically selects agents from existing AI‑scientist hosts during execution, assigns scoped work contracts, and provides artifact‑grounded verification. An exploratory evaluation on ResearchClawBench across three hosts—Agent Laboratory, EvoScientist, and ARK—shows that runtime selection improves performance, while adding contracts and verification can reduce scores depending on the host. The study highlights the potential and limitations of runtime coordination under budget constraints.