← Back to all news
arXiv Computation and Language By Shuyang Zhang

Completeness and Cost of Retrieval Structures for Multi-Tool Requests in LLM Agents

Read the original on arXiv Computation and Language →

The paper evaluates six retrieval structures for multi‑tool requests in LLM agents, measuring how many required tools appear in the top‑k results and the associated LLM calls, tokens, and latency. Across two testbeds—345 AppWorld queries and 103 MCP‑Bench tasks—none of the untrained, single‑LLM‑call structures completes more than 10% of AppWorld queries at k = 16. A trained set‑level retriever and an LLM‑navigated hierarchy achieve higher completeness (up to 58% on MCP‑Bench) but at different costs and failure modes, with complete retrieval sets leading to 100% task success in a downstream AppWorld agent test.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Jul 30

Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents

arXiv:2607. 27083v1 Announce Type: new Abstract: As LLM agents increasingly depend on diverse external services such as search engines, databases, and connectors, agent harnesses face a fundamental tool-selection challenge: acquiring too few tools leaves the task under-informed, while too many adds cost, context load, and privacy exposure.

By Yicheng Feng, Yan Zhang, Yan Cheng, Wei Qi
llmsagentsfine-tuningbenchmarkssafety
More like this →
arXiv AI
Aug 25

Enrich-Retrieve-Rank: Scaling Capability Discovery Beyond In-Context Routing

The paper introduces Enrich‑Retrieve‑Rank, a scalable method for discovering capabilities in large agent ecosystems. It replaces in‑context routing with an offline enrichment step that converts sparse metadata into searchable profiles, followed by an online retrieve‑then‑rank pipeline that returns a ranked shortlist without invoking candidates. Experiments show that as the number of capabilities grows from 10 to 7,278, the new approach maintains higher top‑1 accuracy and reduces cost by 70× compared to full‑context baselines.

By Nazib Sorathiya, Daniel Zhang, Bardiya Akhbari
llmsagents
More like this →
arXiv AI
Aug 26

Hybrid Semantic Tool Discovery for Enterprise MCP Gateway: Architecture and Implementation

Hybrid Semantic Tool Discovery for Enterprise MCP Gateway presents SCOUT, a system that addresses two major challenges in large language model (LLM) agent tool usage: a context‑engineering bottleneck and a tool discoverability barrier. SCOUT reframes tool exposure as a context‑selection problem, injecting only relevant tools into the model’s context window and providing two MCP meta‑tools—tool_search and execute_tool—to perform hybrid retrieval via BM25 and dense vector search. In production at PayPal, SCOUT cuts MCP tool‑token consumption by 99%, dramatically reducing per‑query inference cost while remaining model‑agnostic and requiring no client‑side changes.

By Olympia Saha, Amy Wang, Srinivasan Manoharan
llmsragagents
More like this →