SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond Markdown
Read the original on arXiv AI →SkillScriptBench is a 350‑task benchmark that evaluates how well large language models can self‑evolve executable agent skill packages by distinguishing between documentation repair, script repair, and preservation. The benchmark draws from 100 real GitHub skill packages, creating 150 repair tasks with injected script faults and a controlled track of 200 tasks that test clean, documentation‑faulty, script‑faulty, and combined‑faulty states. Experiments show that AST‑Guided Skill Revision—using abstract syntax trees to target relevant code locations—improves repair success by up to 27.7% and yields more consistent outcomes across repeated runs. whyItMatters":"The benchmark provides a systematic way to measure and improve the reliability of automated skill package maintenance, which is essential for deploying robust LLM agents in real‑world applications."
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.