Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2607. 16900v1 Announce Type: new Abstract: Training API-calling large language model (LLM) agents demands massive amounts of high-quality trajectories.
UniToolCall introduces a unified framework for tool-use in large language model agents, standardizing toolset construction, dataset generation, and evaluation. The framework aggregates over 22,000 tools and creates a hybrid training corpus of more than 390,000 instances by combining ten public datasets with synthetically generated, structurally controlled trajectories. It models diverse interaction patterns—single‑hop vs. multi‑hop, single‑turn vs. multi‑turn, serial vs. parallel execution—and adds an Anchor Linkage mechanism to enforce cross‑turn dependencies, while converting seven public benchmarks into a common Query–Action–Observation–Answer format for fine‑grained evaluation.
The paper introduces KOPA-Bench, a benchmark of 145 real-world tasks that evaluate multi-step tool‑calling over Korean open public APIs. It also presents EDGE, a data‑synthesis method that builds an execution‑grounded dynamic graph to generate executable multi‑step trajectories, and shows that a fine‑tuned 9B model performs nearly as well as an untuned 27B model on KOPA‑Bench and the BFCL benchmark.
The paper introduces Tool Primitives, a design that replaces rigid API schemas with natural language interfaces for tool calling, enabling seamless inter-tool communication. It builds ToolFace, a repository of over 25,000 functions that LLMs can dynamically retrieve, and HEART, a harness engineering framework that orchestrates tool use with planning, routing, and verification. Experiments show HEART outperforms fine‑tuned models and leading commercial LLMs while cutting API costs by up to 85%.
Large language models are deployed as long-horizon agents that compose diverse tools across MCP services into executable workflows. Existing MCP environment-scaling methods are data-centric: they firs...
The paper introduces RobustBench‑TC, a benchmark that tests tool‑use language agents under 22 real‑world perturbations affecting observations, actions, rewards, and transitions. Experiments on 21 models show that observation noise has minimal impact, while reward‑relevant and transition noise drastically reduce accuracy, and model size alone does not mitigate these gaps. The authors propose ToolRL‑DR, a domain‑randomized reinforcement‑learning recipe that improves robustness, retaining most clean accuracy and narrowing performance gaps even on unseen transition perturbations.