arXiv:2607. 04686v1 Announce Type: cross Abstract: Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
ToolRobustBench is a stage-wise diagnostic benchmark designed to evaluate and diagnose failures in tool‑calling agents, which are large language models that select tools, provide structured arguments, and interpret tool feedback. The benchmark aligns four perturbation families—tool‑interface, user‑intent, tool‑output/observation, and runtime‑environment—with the tool‑use pipeline, attributing failures to specific stages such as tool selection, schema grounding, argument binding, and feedback handling. Experiments across 15,456 instances, 7 models, and 16 local tools reveal that while overall performance is high, robustness degrades significantly, especially under tool‑output/observation perturbations, and mixed‑family perturbations produce non‑additive failure patterns.
arXiv:2606. 15508v1 Announce Type: new Abstract: Tool-augmented large language model agents increasingly operate over large tool libraries, but existing evaluations often focus on whether a model can call a tool correctly rather than how the visible tool menu shapes reliability, efficiency, and safety-relevant risk exposure.
arXiv:2609.37315v1 Announce Type: cross Abstract: Tool-using agents are entering settings where a wrong action carries real cost, and the benchmarks certifying them grade what each simulated tool cal...
arXiv:2608. 04719v1 Announce Type: new Abstract: Agent evaluations tell us that a model picked the wrong tool, but rarely why.
arXiv:2607. 14108v1 Announce Type: cross Abstract: This paper introduces tool efficiency, a new quantitative metric to evaluate the rate of useful tool calls in an LLM agent trajectory.
Agent evaluations tell us that a model picked the wrong tool, but rarely why. We introduce canary tools: diagnostic probe tools planted in an agent's Model Context Protocol (MCP) tool set, each engineered to probe one specific tool-selection weakness.