arXiv AI
Aug 18
When Agents Coordinate: Measuring Coordination in Multi-Agent AI Coding
arXiv:2608. 16801v1 Announce Type: new Abstract: We study how teams of AI coding agents coordinate while solving programming tasks.
Scaling Discovery through Test-Time Communication
The paper demonstrates that test‑time communication among agents can significantly outperform independent parallel attempts on complex tasks. In experiments on the ARC‑AGI‑3 benchmark, a team of $k$ communicating agents matched the success rate of $4k$ independent agents, with the advantage growing as the team size increased. The study also shows that communication enables solving tasks that no single agent can solve, and that these benefits transfer to research‑oriented problems such as polyomino packing and MNIST classifier compression, where communicating agents surpassed prior best scores.
Related stories
arXiv Computation and Language
Sep 15
When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis
arXiv:2609.15309v1 Announce Type: new Abstract: Large language model (LLM) agents allocate test-time compute adaptively as they revise solutions, use tools, explore alternatives, and decide when to s...
arXiv AI
Aug 28
DIANOIA: Diagnostic Decomposition and Joint Optimization for Multi-Agent Reasoning
DIANOIA introduces a diagnostic framework for multi‑agent large language model systems, decomposing reasoning gain into three measurable channels—coverage, fidelity, and synthesis. The protocol identifies bottleneck channels for a given task and implements a corresponding multi‑agent system with role‑diverse proposers, execution‑grounded verification, and iterative synthesis. Experiments on GSM8K, AIME‑2025, MBPP, and BFCL‑SP show that DIANOIA outperforms strong baselines, achieving significant token savings and accuracy gains while accurately pinpointing the critical channels.
arXiv AI
Jun 26
Life After Benchmark Saturation: A Case Study of CORE-Bench
arXiv:2606. 26158v1 Announce Type: new Abstract: When a benchmark's accuracy saturates, it is often retired and replaced with a more challenging version.
arXiv AI
Sep 23
Self-Organizing Agent Teams Learn to Reason Together
arXiv:2609.22682v1 Announce Type: new Abstract: Collective intelligence depends not only on what team members know, but also on how they organize their work. When the structure of a solution is unkno...
arXiv AI
1d ago
An Exact Generate - Transform Decomposition of Small-LLM Team Scaling Across Orchestration Architectures
The paper investigates how scaling a team of small language‑model agents affects performance across different orchestration architectures. By testing eight architectures on five short‑answer benchmarks and an executable‑code benchmark, it finds that team scaling yields large gains on arithmetic word‑problem tasks but only modest improvements on multiple‑choice and code generation tasks, with no single architecture dominating all tasks. The authors explain these patterns using a generate‑transform decomposition that separates coverage and transformation effects, showing that arithmetic tasks benefit from both coverage and critic‑guided transformation, while other tasks are limited by saturation or poor conversion.
arXiv AI
Aug 25
Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents
arXiv:2608.22191v1 Announce Type: new Abstract: Software-engineering agents solve repository-level tasks through long, stochastic tool-use trajectories, and repeated attempts often find fixes missed...
arXiv AI
Aug 11
The Collaboration Gap: Exploration and Benchmarking of Open-World Agentic Cooperation
arXiv:2511. 02687v2 Announce Type: replace Abstract: The trajectory of AI development suggests that we will increasingly rely on agent-based systems powered by language models, composed of independently developed agents with different information, privileges, and tools.
arXiv AI
Jun 24
CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark
arXiv:2409. 11363v2 Announce Type: replace-cross Abstract: AI agents have the potential to aid users on a variety of consequential tasks, including conducting scientific research.
arXiv AI
Jul 1
ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
arXiv:2606. 31174v1 Announce Type: new Abstract: Production large language-model (LLM) agents are increasingly deployed not as lone problem-solvers but as managers: a main model creates specialized subagents, delegates work, and orchestrates their parallel, asynchronous returns through dynamic workflows.
arXiv AI
Jun 30
GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes
arXiv:2606. 28514v1 Announce Type: new Abstract: Multimodal models are increasingly deployed to solve tasks collaboratively with humans or other artificial agents.
arXiv Computation and Language
Sep 23
CONCAT: Consensus- and Confidence-Driven Ad Hoc Teaming for Efficient LLM-Based Multi-Agent Systems
CONCAT is a training‑free framework that improves the efficiency of large language model (LLM) based multi‑agent systems by clustering agents according to their initial answers and selecting cluster leaders based on confidence. It uses a Theory‑of‑Mind‑inspired heuristic to predict collaboration benefits between leaders, then prunes communications to form an ad‑hoc network that reduces latency. Experiments on three LLMs and benchmarks show up to 2.02× higher accuracy/latency ratio than LLM‑Debate and a 50.1% latency reduction on Qwen2.5‑14B‑Instruct without task‑specific training.