Hugging Face Trending Papers
RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
6d ago
RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents
RideWay is a new benchmark that evaluates ride‑hailing language agents not just on task completion but on interaction efficiency. It introduces the Efficiency Utility metric, which penalizes agents for excessive tool calls and user‑facing turns relative to a task‑specific reference effort, with human preferences used to calibrate the penalties. Across 58 tasks and 24 models, the metric shows that extra dialogue is penalized more heavily than extra tool use, and it achieves high accuracy in distinguishing trajectories that differ in turns but struggles when differences are only in tool calls.
arXiv AI
Aug 7
Matching Matters: A Fair Quality-Efficiency Benchmark for Command-Line Agents
arXiv:2606. 21140v2 Announce Type: replace-cross Abstract: Rapid advances in large language models have improved the task-solving capabilities of command-line-interface (CLI)-based agents, whose CLIs determine how models invoke tools, maintain interaction history, and recover from failures.
arXiv AI
Sep 15
Utility-Guided Agent Orchestration for Efficient LLM Tool Use
arXiv:2603.19896v2 Announce Type: replace Abstract: Tool-using large language model (LLM) agents often face a fundamental tension between answer quality and execution cost. Fixed workflows are stable...
arXiv AI
Sep 1
AsyncTool: Evaluating the Asynchronous Function Calling Capability under Multi-Task Scenarios
arXiv:2605.27995v3 Announce Type: replace Abstract: Large language model (LLM)-based agents have shown strong capabilities in using external tools to solve complex tasks. However, existing evaluation...
arXiv AI
Sep 10
DAREBench: Deployment-Aware and Reliable Evaluation of Models as Agents
arXiv:2609.06059v1 Announce Type: new Abstract: As large language models evolve from question-answering systems into general-purpose agents, evaluation must move beyond static answer correctness to a...
arXiv AI
Sep 15
MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use
arXiv:2512.24565v4 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly serving as autonomous agents, and their utilization of external tools via the Model Context Protocol...