Bench2Robust is a framework that augments existing tool‑using LLM benchmarks with scenario‑controlled solvability, adding episodes that require retrying the original path, switching to a fallback tool, or recognizing that no viable path remains. It separates recovery‑policy quality from randomness in failure persistence, and across 7 models and 10 task slices, 69 of 70 model‑subset pairs degrade under tool failures by up to 46.7 percentage points. The authors introduce Bayesian Tool Memory (BTM) for structured runtime recovery context and use reinforcement learning (RL) for learned recovery behavior, achieving up to a 20.7‑point improvement over the base model without sacrificing clean performance.
arXiv:2608. 11977v1 Announce Type: new Abstract: Tool-using LLM agents are commonly trained and evaluated in environments where tool calls succeed reliably, yet deployed tools can fail transiently, persistently, or silently.
The paper introduces FRESH, a Failure-aware Retrieval framework that uses Experience-Structured Heterogeneous graphs to transform past successes and failures into structured external memory for tool‑using agents. By explicitly modeling dependencies among tasks, actions, errors, repairs, and execution conditions, FRESH enables frozen language models to reuse reliable strategies, avoid recurring failures, and make safer decisions in stateful tool interactions. Experiments on τ‑Bench and AppWorld with multiple open‑source models demonstrate that FRESH consistently improves task success and tool‑use reliability compared to no‑memory agents and other memory‑based baselines.
arXiv:2610.27954v1 Announce Type: new Abstract: Recent advances in large (vision) language models have enabled computer-use agents that automate digital workflows via fine-grained keyboard and mouse...
The paper introduces RobustBench‑TC, a benchmark that tests tool‑use language agents under 22 real‑world perturbations affecting observations, actions, rewards, and transitions. Experiments on 21 models show that observation noise has minimal impact, while reward‑relevant and transition noise drastically reduce accuracy, and model size alone does not mitigate these gaps. The authors propose ToolRL‑DR, a domain‑randomized reinforcement‑learning recipe that improves robustness, retaining most clean accuracy and narrowing performance gaps even on unseen transition perturbations.
ParaRecover is a new process-level benchmark designed to evaluate error localization and recovery in multi-turn parallel tool-use agents. It contains 10,626 instances across two difficulty levels, built on a taxonomy of 14 error types that cover planning dependencies, tool selection, and argument matching. The benchmark introduces the SDE rubric, which assesses structural integrity, diagnostic reasoning, and evolutionary strategy during agent execution, and demonstrates that it can guide improvements in agents’ reflective recovery capabilities.
The paper introduces Growing Harness, a training method that transforms recurring control logic in large language model agents into reusable executable code, reducing reliance on the model for task-specific decisions. By using strategy-free scaffolds, failure-guided code repair, and success-first gating, the approach learns a shared harness that improves performance across multiple benchmarks and model sizes. Experiments on BrowseComp-Plus and WebArena-Verified show significant gains in success rates and substantial reductions in LLM calls and inference cost compared to traditional tool‑calling agents.
arXiv:2610.16333v1 Announce Type: new Abstract: As AI agents increasingly act through external tools, tool results are no longer simple outputs but execution artifacts that must remain correctly asso...
arXiv:2607. 11126v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly rely on external tools served by shared providers and accessed by heterogeneous downstream agents.
arXiv:2608. 03403v1 Announce Type: new Abstract: The performance bottleneck of agents is increasingly shifting from model capability to the robustness of their execution processes.
RATED (Reliability‑Aware Tool Retrieval via Execution Dynamics) is a retrieval framework that blends robust semantic matching with dynamic reliability adaptation for AI agents using external tools. It uses a frozen dual‑encoder with contrastive fine‑tuning for semantic relevance and a lightweight contextual bandit module that re‑ranks tool candidates based on real‑time execution feedback, allowing rapid adaptation to runtime failures without costly retraining. The EcoAgent benchmark, comprising 5,172 real‑world tools and over 210,000 query‑tool pairs, shows that RATED improves mean reciprocal rank by 52.2% over the ToolLLM API retriever and cuts fault selection from 86.2% to 2.9% during severe API outages, with negligible latency overhead.
arXiv:2606. 01416v1 Announce Type: new Abstract: Tool-augmented large language model (LLM) agents rely on orchestration layers that coordinate planning, retrieval, tool invocation, validation, memory, and recovery.
arXiv:2610.07898v1 Announce Type: new Abstract: Repository-level software engineering (SWE) is a challenging long-horizon setting: agents must reason over extended interactions, use tools, and adapt...