Hugging Face Blog

We got local models to triage the OpenClaw repo for FREE!*

arXiv Machine Learning
Jun 11

Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks

arXiv:2606. 12344v1 Announce Type: new Abstract: General-purpose agents such as OpenClaw are increasingly used as autonomous tool users, but their coding ability is difficult to measure under SWE-bench: a generic agent does not by itself satisfy the clean Docker workspace, patch, and prediction contract required for scoring.

By Mengyu Zheng, Kai Han, Boxun Li, Haiyang Xu, Yuchuan Tian, Wei He, Hang Zhou, Jianyuan Guo, Hailin Hu, Lin Ma, Chao Xu, Guohao Dai, Lixue Xia, Yunchao Wei, Yunhe Wang, Yu Wang
Hugging Face Trending Papers
Jun 10

Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks

General-purpose agents such as OpenClaw are increasingly used as autonomous tool users, but their coding ability is difficult to measure under SWE-bench: a generic agent does not by itself satisfy the clean Docker workspace, patch, and prediction contract required for scoring. We introduce Claw-SWE-Bench, a multilingual SWE-bench-style benchmark and adapter protocol that makes heterogeneous agent harnesses, or claws, comparable under fair settings including a fixed prompt, runtime budget, workspace contract, patch extraction procedure, and evaluator.

arXiv AI
Sep 17

After the Party: Growth, Governance, and Security Scanning in the OpenClaw Agent Skill Ecosystem

The paper examines the rapid growth of the OpenClaw AI agent’s public skill registry, noting a near doubling of the observable stock in 91 days and a concentration of activity in a short period. It finds that only a small fraction of skills receive significant attention—most have no stars or comments—while a large portion contains privilege‑evident code. Automated security scanners show low agreement and sensitivity, indicating that current tools are insufficient for reliable governance of fast‑expanding agent‑skill ecosystems.

By Yunpeng Xiong, Ting Zhang