RideWay is a new benchmark that evaluates ride‑hailing language agents not just on task completion but on interaction efficiency. It introduces the Efficiency Utility metric, which penalizes agents for excessive tool calls and user‑facing turns relative to a task‑specific reference effort, with human preferences used to calibrate the penalties. Across 58 tasks and 24 models, the metric shows that extra dialogue is penalized more heavily than extra tool use, and it achieves high accuracy in distinguishing trajectories that differ in turns but struggles when differences are only in tool calls.
By Qingnuan Han, Boli Fang, Mingzhi Hou, Claire Liu
arXiv:2610.23707v1 Announce Type: new
Abstract: A multi-step agent task fails when a single tool it needs is missing, yet the tools an agent may need are too many to list: the roughly 8,200 tools on...
By Sam Partee, River Stevens, Thierry Damiba, Valerie Fanelle
arXiv:2606. 21140v2 Announce Type: replace-cross Abstract: Rapid advances in large language models have improved the task-solving capabilities of command-line-interface (CLI)-based agents, whose CLIs determine how models invoke tools, maintain interaction history, and recover from failures.
By Han Chi, Jiaxin Qi, Yan Cui, Baisheng Lai, Jianqiang Huang
arXiv:2610.02330v1 Announce Type: new
Abstract: Large language models (LLMs) rely on long-horizon tool invocation sequences for complex tasks, where each invocation can alter the task state and condi...
By Yu Li, Zheng Zhang, Xin Liu, Shengtian Yang, Guangfeng Cai, Lei Feng
AgentBaton explores how capable and efficient language models can collaborate on long‑horizon agentic tasks by handoffs, where one model takes over from another. The study finds that handoffs from a capable to an efficient model at appropriate stages preserve or improve accuracy while significantly cutting cost, whereas the reverse handoff is less effective. AgentBaton, a lightweight detector, learns the optimal handoff point and achieves an average API cost reduction of 65.9% across 11 model pairs while largely maintaining task accuracy, even gaining up to 8.8 percentage points over capable‑only execution.
By Junlong Li, Jindong Jiang, Yonggan Fu, Jiarui Yao, Hanrong Ye, Wonmin Byeon, Peter Belcak, Hao Zhang, Pavlo Molchanov
arXiv:2603.19896v2 Announce Type: replace
Abstract: Tool-using large language model (LLM) agents often face a fundamental tension between answer quality and execution cost. Fixed workflows are stable...
By Boyan Liu, Gongming Zhao, Hongli Xu
arXiv:2605.09252v3 Announce Type: replace
Abstract: Tool-augmented LLM agents tend to call tools indiscriminately, even when the model can answer directly. Each unnecessary call wastes API fees and l...
By Chung-En Sun, Linbo Liu, Ge Yan, Zimo Wang, Tsui-Wei Weng
arXiv:2605.27995v3 Announce Type: replace
Abstract: Large language model (LLM)-based agents have shown strong capabilities in using external tools to solve complex tasks. However, existing evaluation...
By Kou Shi, Ziao Zhang, Shiting Huang, Avery Nie, Zhen Fang, Qiuchen Wang, Lin Chen, Huaian Chen, Zehui Chen, Feng Zhao
arXiv:2609.06059v1 Announce Type: new
Abstract: As large language models evolve from question-answering systems into general-purpose agents, evaluation must move beyond static answer correctness to a...
By Yu Liu, Zhilin Liu, Zhiwei Yang, Shaojie Zhang, Zheyuan Deng, Tingwei Huang, Zhenbo Luo, Lei Jiang, Yanbing Liu, Pei Fu
arXiv:2512.24565v4 Announce Type: replace
Abstract: Large Language Models (LLMs) are increasingly serving as autonomous agents, and their utilization of external tools via the Model Context Protocol...
By Zixiang Liu, Wenrui Liu, Elsie Dai, Wenhan Yu, Lei Yu, Tong Yang, Jinjun Han, Hong Gao
arXiv:2610.19560v1 Announce Type: cross
Abstract: In long-horizon tool-calling tasks, LLM agents can fail not only because of tool-use errors, but also because of ineffective interaction with users....
By Rishitosh Kumar Singh, Amir Saeidi, Shamanthak Hegde, Shrinidhi Kumbhar, Chitta Baral
arXiv:2601. 14192v2 Announce Type: replace Abstract: Recent years have witnessed increasing interest in extending large language models into agentic systems.
By Xiaofang Yang, Lijun Li, Heng Zhou, Tong Zhu, Xiaoye Qu, Yuchen Fan, Qianshan Wei, Rui Ye, Li Kang, Yiran Qin, Daizong Liu, Qi Li, Ning Ding, Siheng Chen, Jing Shao