arXiv AI

ToolSense: A Diagnostic Framework for Auditing Parametric Tool Knowledge in LLMs

arXiv:2606. 12451v1 Announce Type: new Abstract: Large language models deployed as agents over large tool catalogs face a critical tool-retrieval bottleneck.

arXiv AI

Jun 3

Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisition

arXiv:2606. 03657v1 Announce Type: new Abstract: Large language models for code generation often need to use APIs that are absent from their pretraining data.

By Jinnuo Liu, Yue Peng, Jinhan Niu, Hongyi Wen

llms agents fine-tuning benchmarks

arXiv AI

Jul 28

TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs

arXiv:2607. 22639v1 Announce Type: new Abstract: Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search.

By Sai Shruthi Sistla, Ashutosh Hathidara, Christopher Toukmaji, Mayank Shrivastava, Karthikeyan Asokkumar

llms rag fine-tuning

arXiv AI

Jul 29

Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction

arXiv:2607. 25718v1 Announce Type: cross Abstract: Large language model (LLM) agents increasingly rely on invoking external tools to complete real-world tasks.

By Xinyi Hong, Pinjun Dong, Xinyang Yu, Binyan Jiang

llms agents benchmarks

arXiv AI

Jun 4

QO-Bench: Diagnosing Query-Operator-Preserving Retrieval over Typed Event Tuples

arXiv:2606. 04646v1 Announce Type: cross Abstract: Many real-world questions over business, legal, and scientific corpora are natural-language versions of database-style queries over records latent in text.

By Mengao Zhang, Xiang Yang, Chang Liu, Tianhui Tan, Ke-wei Huang

llms rag nlp benchmarks data-engineering

arXiv AI

Jul 8

PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models

arXiv:2607. 05441v1 Announce Type: cross Abstract: Integrating external tools with Large Language Models (LLMs) has emerged as a promising paradigm for accomplishing complex tasks.

By Lorenzo Molfetta, Giacomo Frisoni, Nicol\`o Monaldini, Gianluca Moro

llms safety

arXiv AI

6d ago

VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policies

arXiv:2608. 12282v1 Announce Type: new Abstract: Agents deployed in enterprise settings must reason across structured APIs and document collections, yet existing benchmarks evaluate these capabilities in isolation.

By Ankita Rajaram Naik, Anupama Murthi, Benjamin Elder, Siyu Huo, Raavi Gupta, Abhinav Jain, Praveen Venkateswaran, Abdulhamid Adebayo, Danish Contractor

agents benchmarks

arXiv AI

Jun 6

CTIConnect: A Benchmark for Retrieval-Augmented LLMs over Heterogeneous Cyber Threat Intelligence

arXiv:2510. 11974v2 Announce Type: replace-cross Abstract: Cyber Threat Intelligence (CTI) is foundational to modern cybersecurity, enabling organizations to proactively defend against evolving threats.

By Yutong Cheng, Yang Liu, Changze Li, Dawn Song, Peng Gao

llms rag benchmarks

arXiv AI

Jun 6

ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents

arXiv:2606. 06284v1 Announce Type: new Abstract: Large language model agents increasingly rely on external tools, but larger tool menus can reduce reliability and efficiency by increasing wrong-tool calls, premature actions, and token cost.

By Rahul Suresh Babu, Laxmipriya Ganesh Iyer

llms agents benchmarks

arXiv AI

Jun 11

FitText: Evolving Agent Tool Ecologies via Memetic Retrieval

arXiv:2605. 02411v2 Announce Type: replace Abstract: A semantic gap separates how users describe tasks from how tools are documented.

By Kyle Zheng, Han Zhang, Renliang Sun, Chenchen Ye, Wei Wang

llms rag agents

arXiv Machine Learning

1d ago

When Tool-Backed Skill Retrieval Fails: Source-Style Collapse in Executable Capability Retrieval

arXiv:2608. 16502v1 Announce Type: new Abstract: Large-scale agents increasingly rely on retrieval to access external capabilities.

By Yiqi Liu, Joseph James, Yang Wang, Chenghao Xiao, Chenghua Lin

agents fine-tuning benchmarks

arXiv AI

Jun 8

NTILC: Neural Tool Invocation via Learned Compression

arXiv:2606. 06566v1 Announce Type: cross Abstract: Agentic tool-calling language models depend on large registries of callable APIs, functions, and local actions.

By Andrew Krikorian, Yayuan Li, Jason J. Corso

llms rag agents efficiency

arXiv AI

Jul 22

M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generation

arXiv:2603. 26667v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) turns external documents into evidence for large language models.

By Xu Sun, Tongkai Xu, Baiheng Xie, Li Huang, Qiang Gao, Kunpeng Zhang

llms rag

Related stories