arXiv Machine Learning

Toollery: Scaling LLM Agents to Thousands of Skills and Tools

Toollery is a training‑free framework that compresses candidate lists for large language model agents, enabling efficient selection from thousands of skills and tools. It generates user‑intent queries from each skill or tool specification, builds a retrieval index, and limits online selection to a compact top‑k set before the LLM makes its final decision. Evaluations on the SkillRouter benchmark, BFCL‑V4, and a proprietary smart‑cockpit dataset show that Toollery improves recall and end‑to‑end selection while keeping selection costs bounded.

arXiv AI

Sep 1

SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents

arXiv:2605.05726v2 Announce Type: replace Abstract: As LLM agents are increasingly deployed with large libraries of reusable skills, selecting the right skill for a user request has become a critical...

By Hongcheol Cho, Ryangkyung Kang, Youngeun Kim

llms rag agents fine-tuning benchmarks

arXiv AI

Jul 29

Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction

arXiv:2607. 25718v1 Announce Type: cross Abstract: Large language model (LLM) agents increasingly rely on invoking external tools to complete real-world tasks.

By Xinyi Hong, Pinjun Dong, Xinyang Yu, Binyan Jiang

llms agents benchmarks

arXiv Machine Learning

Jul 21

SkillRouter: Skill Routing for LLM Agents at Scale

arXiv:2603. 22455v5 Announce Type: replace Abstract: Reusable skills let LLM agents package task-specific procedures, tool affordances, and execution guidance into modular building blocks.

By YanZhao Zheng, ZhenTao Zhang, Chao Ma, YuanQiang Yu, JiHuai Zhu, Yong Wu, Tianze Xu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, Gang Yu

llms rag agents benchmarks

arXiv Machine Learning

Aug 18

When Tool-Backed Skill Retrieval Fails: Source-Style Collapse in Executable Capability Retrieval

arXiv:2608. 16502v1 Announce Type: new Abstract: Large-scale agents increasingly rely on retrieval to access external capabilities.

By Yiqi Liu, Joseph James, Yang Wang, Chenghao Xiao, Chenghua Lin

agents fine-tuning benchmarks

arXiv AI

Jul 15

SkillSelect-Serve: QoS-Aware Budgeted Skill Service Recommendation for LLM Agents

arXiv:2607. 00011v2 Announce Type: replace-cross Abstract: Reusable agent skills are emerging as a service-oriented capability layer for Large Language Model (LLM) agents.

By Jingyuan Zheng, Dongjing Wang, Xin Zhang, Hao Chen, Youhuizi Li, Xudong Shen, Haiping Zhang, Butian Huang, Dongjin Yu, Guandong Xu

llms agents

arXiv AI

Jun 12

ToolSense: A Diagnostic Framework for Auditing Parametric Tool Knowledge in LLMs

arXiv:2606. 12451v1 Announce Type: new Abstract: Large language models deployed as agents over large tool catalogs face a critical tool-retrieval bottleneck.

By Ashutosh Hathidara, Sai Shruthi Sistla, Sebastian Schreiber, Sahil Bansal

llms rag agents fine-tuning benchmarks

arXiv AI

Jun 3

SkillDAG: Self-Evolving Typed Skill Graphs for LLM Skill Selection at Scale

arXiv:2606. 03056v1 Announce Type: new Abstract: As LLM agents adopt large skill libraries, selecting the right subset becomes a structural problem rather than a similarity-matching one: skills depend on, conflict with, specialize, or duplicate one another, a structure invisible to both full enumeration and embedding similarity.

By Tong Bai, Zhenglin Wan, Pengfei Zhou, Xingrui Yu, Wangbo Zhao, Yang You, Ivor W. Tsang

llms rag agents diffusion

arXiv AI

Aug 11

From Relevance to Execution Utility: Reward-Aware Dynamic Execution Gating for Skill-Based LLM Agents

arXiv:2608. 09168v1 Announce Type: new Abstract: Agent skills are increasingly used to equip large language model (LLM) agents with reusable procedural knowledge.

By Liang He, Jingbo Wen, Hongyu Gu, Hao Li, Haoyu Wang, Yixiong Chen, Kangning Cui, Xilu Wang

llms agents

arXiv AI

Sep 15

MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use

arXiv:2512.24565v4 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly serving as autonomous agents, and their utilization of external tools via the Model Context Protocol...

By Zixiang Liu, Wenrui Liu, Elsie Dai, Wenhan Yu, Lei Yu, Tong Yang, Jinjun Han, Hong Gao

llms agents benchmarks

arXiv AI

Jul 17

OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios

arXiv:2607. 14989v1 Announce Type: cross Abstract: Large language models are increasingly evolving from text generators into general agents capable of understanding user requests, invoking external tools, and completing complex tasks through interaction.

By Chengyu Shen, Yujie Fu, Gangtao Xin, Yanheng Hou, Wenlong Fei, Guojie Zhu, Jiawei Li, Hongcheng Gao, Runming He, Zhen Hao Wong, Meiyi Qiang, Hao Liang, Zhao Cao, Hao Jiang, Chong Chen, Wentao Zhang

llms agents benchmarks

arXiv Machine Learning

Jul 30

Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents

arXiv:2607. 27083v1 Announce Type: new Abstract: As LLM agents increasingly depend on diverse external services such as search engines, databases, and connectors, agent harnesses face a fundamental tool-selection challenge: acquiring too few tools leaves the task under-informed, while too many adds cost, context load, and privacy exposure.

By Yicheng Feng, Yan Zhang, Yan Cheng, Wei Qi

llms agents fine-tuning benchmarks safety

Hugging Face Trending Papers

Aug 10

From Relevance to Execution Utility: Reward-Aware Dynamic Execution Gating for Skill-Based LLM Agents

Agent skills are increasingly used to equip large language model (LLM) agents with reusable procedural knowledge. Although recent work has substantially improved skill retrieval due to the increasing skill libraries, retrieving a plausible skill bundle does not guarantee that executing it is worthwhile.

llms agents

Related stories