arXiv Machine Learning

Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents

arXiv:2607. 27083v1 Announce Type: new Abstract: As LLM agents increasingly depend on diverse external services such as search engines, databases, and connectors, agent harnesses face a fundamental tool-selection challenge: acquiring too few tools leaves the task under-informed, while too many adds cost, context load, and privacy exposure.

arXiv Machine Learning
Sep 22

Toollery: Scaling LLM Agents to Thousands of Skills and Tools

Toollery is a training‑free framework that compresses candidate lists for large language model agents, enabling efficient selection from thousands of skills and tools. It generates user‑intent queries from each skill or tool specification, builds a retrieval index, and limits online selection to a compact top‑k set before the LLM makes its final decision. Evaluations on the SkillRouter benchmark, BFCL‑V4, and a proprietary smart‑cockpit dataset show that Toollery improves recall and end‑to‑end selection while keeping selection costs bounded.

By Xiangxi Tian, Ran Guan
arXiv Machine Learning
Jun 11

Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks

arXiv:2606. 12344v1 Announce Type: new Abstract: General-purpose agents such as OpenClaw are increasingly used as autonomous tool users, but their coding ability is difficult to measure under SWE-bench: a generic agent does not by itself satisfy the clean Docker workspace, patch, and prediction contract required for scoring.

By Mengyu Zheng, Kai Han, Boxun Li, Haiyang Xu, Yuchuan Tian, Wei He, Hang Zhou, Jianyuan Guo, Hailin Hu, Lin Ma, Chao Xu, Guohao Dai, Lixue Xia, Yunchao Wei, Yunhe Wang, Yu Wang
arXiv AI
Sep 7

Iris: Climbing to the Search Frontier

The paper introduces Iris-mini and Iris-pro, two search agents trained at 35B and 397B parameter scales. They use a novel data pipeline that constructs reverse‑engineered multi‑hop queries from web hyperlinks, filters trajectories, and alternates supervised fine‑tuning with reinforcement learning in a process called SFT‑RL climbing. Evaluations on several benchmarks show that, with inference‑time context management, the agents achieve the best open‑source results in their parameter ranges.

By Ziyuan Liu, Hengqi Liu, Zichuan Wang, Yang Qin, Jiachen Liang, Xu Chu, Shaowei Chen, Yuantao Gu, Mu Chuan
arXiv AI
Aug 25

Enrich-Retrieve-Rank: Scaling Capability Discovery Beyond In-Context Routing

The paper introduces Enrich‑Retrieve‑Rank, a scalable method for discovering capabilities in large agent ecosystems. It replaces in‑context routing with an offline enrichment step that converts sparse metadata into searchable profiles, followed by an online retrieve‑then‑rank pipeline that returns a ranked shortlist without invoking candidates. Experiments show that as the number of capabilities grows from 10 to 7,278, the new approach maintains higher top‑1 accuracy and reduces cost by 70× compared to full‑context baselines.

By Nazib Sorathiya, Daniel Zhang, Bardiya Akhbari
Hugging Face Trending Papers
Aug 19

Pairwise Ranking Outperforms Single-Action RL for Offline Explanation Selection: A Practical Lesson

The paper presents a cost‑effective approach for industrial explainable‑recommendation systems by decoupling explanation generation from selection. Explanations are pre‑generated using six prompt styles and two commodity LLMs, then a lightweight CPU‑resident selector (e.g., LambdaRank) chooses the best one at request time, achieving sub‑100 ms latency without GPUs. Experiments on a 2,958‑pair Google Local subset and a 300‑pair MovieLens‑1M split show that pairwise ranking outperforms single‑action RL methods, while KG‑path selectors achieve near‑perfect unique‑output rates, and the overall end‑to‑end build cost is around $15 on commodity hardware.

Hugging Face Trending Papers
Jun 10

Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks

General-purpose agents such as OpenClaw are increasingly used as autonomous tool users, but their coding ability is difficult to measure under SWE-bench: a generic agent does not by itself satisfy the clean Docker workspace, patch, and prediction contract required for scoring. We introduce Claw-SWE-Bench, a multilingual SWE-bench-style benchmark and adapter protocol that makes heterogeneous agent harnesses, or claws, comparable under fair settings including a fixed prompt, runtime budget, workspace contract, patch extraction procedure, and evaluator.