The paper introduces BenchAgent, a framework for evaluating multi‑agent large language model (LLM) workflows against a single‑agent baseline, BenchAgent Core. Using GPT‑4.1 across ten reasoning, coding, and tool‑use benchmarks, the study compares six multi‑agent configurations, finding that EvoAgent achieves the highest mean pooled accuracy (79.70%) while other agents lag by 1.71–8.76 percentage points. The analysis also highlights trade‑offs in token consumption and accuracy, offering guidance for selecting and optimizing LLM multi‑agent workflows.
arXiv:2606. 05670v1 Announce Type: new Abstract: Does adding more agents help an LLM workflow once compared systems share the same benchmark loader, tool access, answer contract, usage accounting, and trajectory logging?
arXiv:2608. 05519v1 Announce Type: new Abstract: Agent benchmarks usually measure task completion and treat resource use as an auxiliary statistic.
arXiv:2607. 20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or narrow action spaces.
UniACE is a unified framework that standardizes the evaluation of large language model (LLM) agents by representing each benchmark as an instruction–tool–environment triplet and running models through a shared, task‑agnostic harness in isolated runtimes. It preserves native success criteria, offers an offline mode for dynamic‑resource tasks, and standardizes efficiency metrics, execution records, and failure attribution. Applying UniACE to 7 benchmarks across 24 domains and 15 models revealed significant score shifts, ranking reversals, and sensitivity to evidence representation, highlighting the impact of evaluation configuration on reported agent performance.
The paper introduces Humanize MLE Agents (HMA), a workflow that alternates two agents across sessions on a shared workspace, limiting each session to at most k experiments and resetting context between sessions. This approach, modeled as a context‑reset semi‑Markov decision process, outperforms single‑agent baselines, achieving a 78.2% any‑medal rate on 75 Kaggle MLE‑bench tasks—8.0 points higher than the best open‑source baseline and using only a quarter of its time budget. Across six heterogeneous agent pairs, HMA improves the any‑medal rate by an average of 6.4 points, and a supplementary kernel‑optimization case reaches 878 cycles, within 1.6% of the community leaderboard leader.
The paper introduces InFlowOp, a label‑free optimization framework that assigns costs to each decision in a multi‑agent workflow, balancing agent competence against execution time. It determines task granularity and agent assignment before execution and corrects faults during execution using the same cost metric. The authors also present Braid, a benchmark for multi‑agent coordination, and show that InFlowOp outperforms single‑agent baselines by up to 11.97% across various domains.
arXiv:2610.23707v1 Announce Type: new Abstract: A multi-step agent task fails when a single tool it needs is missing, yet the tools an agent may need are too many to list: the roughly 8,200 tools on...
arXiv:2609.01045v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities as powerful components in agentic systems, enabling sophisticated reasoning and...
arXiv:2610.09633v1 Announce Type: new Abstract: The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engine...
arXiv:2605. 27898v2 Announce Type: replace Abstract: As LLMs are increasingly deployed as agents, reliable assessment of their agentic capabilities has become essential.
arXiv:2610.22869v1 Announce Type: new Abstract: LLM agents are increasingly integrated into real-world works, yet they must handle diverse domain-specific tools and complex multi-sub-task problems th...
arXiv:2606. 07682v1 Announce Type: cross Abstract: AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex environments.