← Back to all news
arXiv AI

Folding Pooled Usage Examples into the Tool Index Lowers Retrieval of Unused Tools

The paper investigates how adding past usage examples of tools to a shared tool index—called folding—affects the retrieval of tools that an AI agent has never used. While folding improves retrieval for tools with examples, it significantly reduces the likelihood that an unused tool appears in the top five results on two of three evaluated catalogs, dropping from 73% to 40% in one benchmark. The authors propose a new scoring method, ZoE, which separates documentation and usage scores, and show that ZoE retrieves unused tools more often than a simple router or a constant‑boost rule, suggesting that evaluations of pooled agent experience should distinguish between used and unused tools.

Related stories

arXiv Computation and Language
5d ago

Completeness and Cost of Retrieval Structures for Multi-Tool Requests in LLM Agents

The paper evaluates six retrieval structures for multi‑tool requests in LLM agents, measuring how many required tools appear in the top‑k results and the associated LLM calls, tokens, and latency. Across two testbeds—345 AppWorld queries and 103 MCP‑Bench tasks—none of the untrained, single‑LLM‑call structures completes more than 10% of AppWorld queries at k = 16. A trained set‑level retriever and an LLM‑navigated hierarchy achieve higher completeness (up to 58% on MCP‑Bench) but at different costs and failure modes, with complete retrieval sets leading to 100% task success in a downstream AppWorld agent test.

By Shuyang Zhang
llmsagents
More like this →
arXiv AI
Jun 30

When Does Overlap Help? OSU-Mem and a Cell-Conditional Analysis of Trajectory Memory for LLM Agents

arXiv:2606. 28376v1 Announce Type: cross Abstract: Long-horizon large language model (LLM) agents accumulate interaction trajectories that quickly exceed any practical prompt budget, and existing memory methods either truncate aggressively and lose non-local evidence or retain boilerplate that degrades decision quality.

By Mellow Baixuan Chen, Xiangguo Sun
llmsagentsbenchmarks
More like this →
arXiv AI
Sep 10

Typed Federated Artifacts for the Agentic Web:Sharing Tool-Routing Knowledge Across Frozen,Heterogeneous LLM Agents

The paper proposes typed federated artifacts—schema‑validated objects with per‑field privacy and dispute resolution—to enable tool‑routing knowledge sharing among frozen, heterogeneous LLM agents. By replacing flat text prompts with typed fields, the authors achieve near‑centralized routing performance on StableToolBench while reducing data size to 20 MB JSON per client. The study also highlights that a simple TF‑IDF classifier can outperform LLM routing on labeled benchmarks, indicating limitations in current evaluation methods.

By Abhijit Chakraborty, Ni Trieu, Vivek Gupta
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Oct 12

Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA

KnowForge is a modular pipeline for open‑domain multimodal document question answering that separates retrieval, judging, and reading into distinct agents. It uses a hybrid‑retrieval knowledge‑point index, a Locator agent for searching, a Bridge Judge to enforce a second hop, a Deep Reader that handles text, images, and tables, and a synthesizer with format checking. On the M3DocVQA benchmark it achieves an F1 of 47.5, outperforming prior systems by 11 points, and shows strong gains on MultiHop‑RAG and MMLongBench‑Doc as well.

By Wenlin Lin, Fanzhe Wei
llmsragagentsnlpmultimodalbenchmarks
More like this →