← Back to all news
arXiv AI By Jongho Park, Vasilis Kontonis, Shivam Garg, Akshay Krishnamurthy, Dimitris Papailiopoulos

Scaling Discovery through Test-Time Communication

Read the original on arXiv AI →

The paper demonstrates that test‑time communication among agents can significantly outperform independent parallel attempts on complex tasks. In experiments on the ARC‑AGI‑3 benchmark, a team of $k$ communicating agents matched the success rate of $4k$ independent agents, with the advantage growing as the team size increased. The study also shows that communication enables solving tasks that no single agent can solve, and that these benefits transfer to research‑oriented problems such as polyomino packing and MNIST classifier compression, where communicating agents surpassed prior best scores.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Sep 15

When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis

arXiv:2609.15309v1 Announce Type: new Abstract: Large language model (LLM) agents allocate test-time compute adaptively as they revise solutions, use tools, explore alternatives, and decide when to s...

By Kaiyuan Liu, Qiuyang Mang, Bo Peng, Wenhao Chai, Hanchen Li, Shreyas Pimpalgaonkar, Luke Zettlemoyer, Alex Dimakis, Alvin Cheung
llmsagentsbenchmarks
More like this →
arXiv AI
Aug 28

DIANOIA: Diagnostic Decomposition and Joint Optimization for Multi-Agent Reasoning

DIANOIA introduces a diagnostic framework for multi‑agent large language model systems, decomposing reasoning gain into three measurable channels—coverage, fidelity, and synthesis. The protocol identifies bottleneck channels for a given task and implements a corresponding multi‑agent system with role‑diverse proposers, execution‑grounded verification, and iterative synthesis. Experiments on GSM8K, AIME‑2025, MBPP, and BFCL‑SP show that DIANOIA outperforms strong baselines, achieving significant token savings and accuracy gains while accurately pinpointing the critical channels.

By Yiming Yang, Zhuoyuan Li, Fanxiang Zeng, Hao Fu, Yue Liu
llmsagentsbenchmarks
More like this →
arXiv AI
Jun 26

Life After Benchmark Saturation: A Case Study of CORE-Bench

arXiv:2606. 26158v1 Announce Type: new Abstract: When a benchmark's accuracy saturates, it is often retired and replaced with a more challenging version.

By Nitya Nadgir, Sayash Kapoor, Kangheng Liu, Peter Kirgis, Matilda Orona, Stephan Rabanser, Tilman Bayer, Abhishek Shetty, Yue Ling, Derrick Chan-Sew, Rumi Nakagawa, Saiteja Utpala, Zachary S. Siegel, Arvind Narayanan
agentsbenchmarks
More like this →
arXiv AI
1d ago

An Exact Generate - Transform Decomposition of Small-LLM Team Scaling Across Orchestration Architectures

The paper investigates how scaling a team of small language‑model agents affects performance across different orchestration architectures. By testing eight architectures on five short‑answer benchmarks and an executable‑code benchmark, it finds that team scaling yields large gains on arithmetic word‑problem tasks but only modest improvements on multiple‑choice and code generation tasks, with no single architecture dominating all tasks. The authors explain these patterns using a generate‑transform decomposition that separates coverage and transformation effects, showing that arithmetic tasks benefit from both coverage and critic‑guided transformation, while other tasks are limited by saturation or poor conversion.

By Blaz Bertalanic, Carolina Fortuna
llmsagentsbenchmarks
More like this →