arXiv Computation and Language By Yuxuan Lu, Ziyi Wang, Yingzhou Lu, Yisi Sang, Jiri Gesi, Xianfeng Tang, Yimeng Zhang, Zhenwei Dai, Hui Liu, Hanqing Lu, Chen Luo, Qi He, Benoit Dumoulin, Jing Huang, Dakuo Wang

Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs

Read the original on arXiv Computation and Language →

The Flow has not summarised this story yet — read it at arXiv Computation and Language.

arXiv AI
Sep 3

UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents

UniToolCall introduces a unified framework for tool-use in large language model agents, standardizing toolset construction, dataset generation, and evaluation. The framework aggregates over 22,000 tools and creates a hybrid training corpus of more than 390,000 instances by combining ten public datasets with synthetically generated, structurally controlled trajectories. It models diverse interaction patterns—single‑hop vs. multi‑hop, single‑turn vs. multi‑turn, serial vs. parallel execution—and adds an Anchor Linkage mechanism to enforce cross‑turn dependencies, while converting seven public benchmarks into a common Query–Action–Observation–Answer format for fine‑grained evaluation.

By Yijuan Liang, Xinghao Chen, Yifan Ge, Ziyi Wu, Hao Wu, Changyu Zeng, Wei Xing, Xiaoyu Shen
arXiv AI
Sep 7

Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe

The paper introduces KOPA-Bench, a benchmark of 145 real-world tasks that evaluate multi-step tool‑calling over Korean open public APIs. It also presents EDGE, a data‑synthesis method that builds an execution‑grounded dynamic graph to generate executable multi‑step trajectories, and shows that a fine‑tuned 9B model performs nearly as well as an untuned 27B model on KOPA‑Bench and the BFCL benchmark.

By Dain Kim, Eungi Cho, Kyumin Kim, Shinyeong Noh, Kyuseong Lim
arXiv AI
Sep 3

Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives

The paper introduces Tool Primitives, a design that replaces rigid API schemas with natural language interfaces for tool calling, enabling seamless inter-tool communication. It builds ToolFace, a repository of over 25,000 functions that LLMs can dynamically retrieve, and HEART, a harness engineering framework that orchestrates tool use with planning, routing, and verification. Experiments show HEART outperforms fine‑tuned models and leading commercial LLMs while cutting API costs by up to 85%.

By Haibo Jin, Suijin Wang, Xucheng Yu, Haojing Luo, Haohan Wang
arXiv AI
Oct 12

When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents

The paper introduces RobustBench‑TC, a benchmark that tests tool‑use language agents under 22 real‑world perturbations affecting observations, actions, rewards, and transitions. Experiments on 21 models show that observation noise has minimal impact, while reward‑relevant and transition noise drastically reduce accuracy, and model size alone does not mitigate these gaps. The authors propose ToolRL‑DR, a domain‑randomized reinforcement‑learning recipe that improves robustness, retaining most clean accuracy and narrowing performance gaps even on unseen transition perturbations.

By Xiaolin Zhou, Aojie Yuan, Zheng Luo, Zipeng Ling, Xixiao Pan, Yicheng Gao, Haiyue Zhang, Jiate Li, Shuli Jiang, Prince Zizhuang Wang, Zixuan Zhu, Jinbo Liu, Ryan A. Rossi, Hua Wei, Xiyang Hu