← Back to all news
arXiv AI

Verifying Coordination in Parallel Coding Agents: NP-Bench and a Scheduling Planner

The paper introduces NP‑Bench, a benchmark and a proactive scheduling planner that coordinates parallel large‑language‑model coding agents. By partitioning work scopes and ordering merges ahead of time, the planner improves clean‑integration rates from 1/9 to 9/9 and eliminates merge conflicts, outperforming both no‑coordination and reactive‑detection baselines. It also demonstrates that cross‑session memory can eliminate repeated mistakes and that routing facts to agents does not improve long‑context accuracy at scale.

Related stories

arXiv AI
6d ago

Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams

The paper investigates how coordination among AI agents serving different users degrades performance compared to a single coordinating agent. Across five advanced models and 77 scenarios in four shared-resource environments—API key budgets, clinic calendars, personal assistant bookings, and merge queues—the study finds that multi‑agent teams consistently underperform, sometimes collapsing entirely, and that even with communication channels coordination overhead remains significant. The authors identify specific failure modes such as stalling, action overriding, and claim fabrication, and propose environment‑specific mitigations like team leads and procedural instructions, while releasing the MAMUBench benchmark for future research.

By Sahan Paliskara, Nattaput Namchittai, Andrew Lampinen
agents
More like this →
arXiv AI
Jun 9

Benchmarking Open-Ended Multi-Agent Coordination in Language Agents

arXiv:2606. 08340v1 Announce Type: new Abstract: As language models are increasingly deployed as autonomous agents, they must coordinate with others over long horizons in open-ended interactive tasks.

By Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker, Alexander Rutherford, Davide Paglieri, Aidan Scannell, Henry Gouk, Elliot J. Crowley, Tim Rockt\"aschel, Amos Storkey
llmsagentsbenchmarks
More like this →
arXiv AI
Jul 14

AgentAbstain: Do LLM Agents Know When Not to Act?

arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.

By Xun Liu, Yi Evie Zhang, Vira Kasprova, Parisa Rabbani, Pardis Sadat Zahraei, Tianyu Zhang, Ali Ebrahimpour-Boroojeny, Varun Chandrasekaran
llmsagentsbenchmarks
More like this →
arXiv AI
Aug 26

AgentRoom: Concurrent Multi-Agent Coding in a CRDT-Backed Shared Workspace

AgentRoom introduces a real‑time collaborative editing protocol that enables concurrent coding by multiple large language model agents within a CRDT‑backed shared workspace. By providing file‑level claim, status, and broadcast tools, it allows agents to coordinate directly rather than relying on serial phase handoffs or independent sampling. Experiments with five frontier coding‑CLI models show that AgentRoom reduces task abandonment and run‑to‑run variation compared to solo or parallel‑merge approaches, highlighting the importance of coordination over mere parallelism.

By Seonglae Cho, Donghyun Lee
llmsagents
More like this →
arXiv AI
Sep 25

Policy as Code: A Coroutine-Bridge Harness for Fast-Reasoning Reliability on CAR-bench

The paper introduces a coroutine-bridge harness that lets a language model emit a Python program to manage tool calls in the CAR-bench evaluation. By decoupling model invocations from tool round-trips, the approach reduces model calls to a median of two per task while maintaining seven agent turns, achieving a median latency of 1.8 s on a Cerebras gpt‑oss‑120b. The harness achieved 60.0 % Pass³ on the official hidden evaluation, outperforming the baseline by 4.5× and matching frontier-model agents on GPT‑5.5, all while keeping the prompt largely cached and minimizing input compute.

By Ivan Matveev
llmsagents
More like this →