arXiv AI June 16, 2026 By Rahul Suresh Babu, Laxmipriya Ganesh Iyer

ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents

arXiv:2606. 15508v1 Announce Type: new Abstract: Tool-augmented large language model agents increasingly operate over large tool libraries, but existing evaluations often focus on whether a model can call a tool correctly rather than how the visible tool menu shapes reliability, efficiency, and safety-relevant risk exposure.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 6

ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents

arXiv:2606. 06284v1 Announce Type: new Abstract: Large language model agents increasingly rely on external tools, but larger tool menus can reduce reliability and efficiency by increasing wrong-tool calls, premature actions, and token cost.

By Rahul Suresh Babu, Laxmipriya Ganesh Iyer

llms agents benchmarks

arXiv AI

Jul 7

ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents

arXiv:2607. 04686v1 Announce Type: cross Abstract: Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails.

By Harsh Soni

llms agents benchmarks

arXiv AI

Sep 15

MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use

arXiv:2512.24565v4 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly serving as autonomous agents, and their utilization of external tools via the Model Context Protocol...

By Zixiang Liu, Wenrui Liu, Elsie Dai, Wenhan Yu, Lei Yu, Tong Yang, Jinjun Han, Hong Gao

llms agents benchmarks

arXiv AI

Jun 16

GIST-CMTF: Goal-State Inference for Causal Minimal Tool Filtering in LLM Agents

arXiv:2606. 16813v1 Announce Type: new Abstract: Tool-augmented LLM agents rely on runtime filtering to decide which tools should be visible at each step.

By Rahul Suresh Babu, Rohit Shukla

llms agents

arXiv AI

Jul 3

A Unified Framework for the Evaluation of LLM Agentic Capabilities

arXiv:2605. 27898v2 Announce Type: replace Abstract: As LLMs are increasingly deployed as agents, reliable assessment of their agentic capabilities has become essential.

By Pengyu Zhu, Lijun Li, Yaxing Lyu, Qianxin Luo, Jingyi Yang, Yi Liu, Tingfeng Hui, Xinyu Yuan, Li Sun, Sen Su, Jing Shao

llms agents benchmarks safety

arXiv AI

Aug 6

Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools

arXiv:2608. 04719v1 Announce Type: new Abstract: Agent evaluations tell us that a model picked the wrong tool, but rarely why.

By Atul Anand, Sourav Chattaraj

llms agents benchmarks safety

One email a morning, machine-written

Related stories

ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents

ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents

MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use

GIST-CMTF: Goal-State Inference for Causal Minimal Tool Filtering in LLM Agents

A Unified Framework for the Evaluation of LLM Agentic Capabilities

Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools