arXiv AI
AsyncTool: Evaluating the Asynchronous Function Calling Capability under Multi-Task Scenarios
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 2
UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities
UniACE is a unified framework that standardizes the evaluation of large language model (LLM) agents by representing each benchmark as an instruction–tool–environment triplet and running models through a shared, task‑agnostic harness in isolated runtimes. It preserves native success criteria, offers an offline mode for dynamic‑resource tasks, and standardizes efficiency metrics, execution records, and failure attribution. Applying UniACE to 7 benchmarks across 24 domains and 15 models revealed significant score shifts, ranking reversals, and sensitivity to evidence representation, highlighting the impact of evaluation configuration on reported agent performance.
arXiv AI
Jun 16
Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving
arXiv:2603. 18897v2 Announce Type: replace-cross Abstract: LLM-powered agents execute tasks through a sequential loop of model generation and tool execution.
arXiv AI
Jul 28
E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios
arXiv:2607. 23722v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed as agents that interact with stateful environments over multiple steps: gathering hidden information, composing tool calls, and committing state changes.
arXiv AI
Aug 26
PeakBench: Benchmarking Resource-Aware Tool Invocation in LLM Agents
PeakBench is a new benchmark designed to evaluate how large language model agents invoke multiple tools while respecting resource constraints and parallel execution. It provides executable multi‑tool workflows with dependency annotations and measured resource profiles, and introduces a two‑part evaluation framework that separates logical planning from physical scheduling. The study shows that strong logical planning alone does not guarantee safe or efficient execution, and that providing resource information can reduce overflows and improve utilization.
arXiv AI
Sep 10
Substrate-Portable Execution for Production LLM Workflows
arXiv:2609.06128v1 Announce Type: new Abstract: Production LLM agents execute tool-calling loops, retrieval chains, and compositional workflows in multiple modes, yet execution semantics are often co...
arXiv AI
Jul 14
AgentAbstain: Do LLM Agents Know When Not to Act?
arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.