arXiv AI October 26, 2026 By Yize Cheng, Chenrui Fan, Mahdi JafariRaviz, Keivan Rezaei, Soheil Feizi

Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

The paper introduces a model‑adaptive definition of tool necessity that accounts for each LLM’s empirical performance, contrasting it with observed tool‑call behavior across four models on arithmetic and factual QA datasets. It finds substantial mismatches—26.5‑54.0% for arithmetic and 30.8‑41.8% for factual QA—between when tools are needed and when they are actually used. By probing hidden states, the authors show that the main source of mismatch lies in the transition from internal cognition (recognizing tool necessity) to action (making a tool call), revealing a knowing‑doing gap in LLM tool use.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Sep 15

When Tools Get in the Way: The Effect of Unnecessary Tool Availability on LLM Answering

arXiv:2609.14157v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed with external tools that extend what they can do beyond their own knowledge. Tools help on tasks...

By Saanvi Paturi, Arsen Kenzhebayev, Arham Sethi, Vyas Raina, Ivaxi Sheth, Vatsal Raina

llms

arXiv Computation and Language

Sep 16

Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act

The paper investigates how reinforcement learning can cause large language model agents to adopt shortcut policies for tool use, relying on superficial prompt cues rather than actual task needs. By creating synthetic environments that mix factual QA and math reasoning, the authors show that agents often invoke tools when cues are present, even when those tools are unnecessary, with spurious invocation rates rising up to 39%. They find that shortcut learning occurs mainly when agents have already mastered the target tool and that semantic alignment between cues and tools amplifies the effect. To counter this, they propose a dense, decision-level reward where an LLM judge assesses tool necessity, which reduces cue-driven tool use while maintaining performance.

By Yiwei Yang, Haoxiang Zhang, Bingbing Wen, Yao Lu, Yuchen Wu, Lei Zhang, Julian McAuley, Pan Lu, Bill Howe

llms agents nlp reinforcement-learning safety

arXiv AI

Sep 15

ZEBRAARENA: A Diagnostic Simulation Environment for Studying Reasoning-Action Coupling in Tool-Augmented LLMs

arXiv:2603.18614v2 Announce Type: replace Abstract: Tool-augmented large language models (LLMs) must tightly couple multi-step reasoning with external actions, yet existing benchmarks often confound...

By Wanjia Zhao, Ludwig Schmidt, Yejin Choi, James Zou, Vidhisha Balachandran, Lingjiao Chen

llms agents benchmarks

arXiv Machine Learning

Oct 8

How Do Agentic LLMs Decide to Call Tools? A Tool-Call Vector Shaped by Suppression

arXiv:2610.09624v1 Announce Type: cross Abstract: Tool calling, invoking external tools on demand, is central to agentic LLMs, yet the mechanism that decides whether a model calls a tool or responds...

By Xijie Gong, Tingxu Han, Jiahao Zhang, Wei Song, Ziqi Ding, Hanqi Yan, Youcheng Sun, Lijie Hu

llms agents

arXiv AI

Jul 7

ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents

arXiv:2607. 04686v1 Announce Type: cross Abstract: Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails.

By Harsh Soni

llms agents benchmarks

Hugging Face Trending Papers

Sep 28

Self-Evolving Agents via Likelihood-Guided Tool-Space Optimization

Self-evolving agents can continually improve their behavior, while tools define the executable action space through which they interact with the environment. However, exposing the full tool library to...

agents benchmarks