arXiv AI

Right Family, Wrong Skill: Evaluating Risk Exposure in Agent Skill Retrieval

arXiv AI
3d ago

One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents

The paper investigates how co-installed coding‑agent skills can conflict, causing the model to choose a similar skill over the intended one based solely on name and description. An empirical study of 20,947 repositories uncovered 822,109 candidate skill pairs, with 312 confirmed conflicts tested across three models, revealing that nearly one in four installed skills is overridden by a similar skill. Key findings include that conflicts often involve normative skills, that the installed skill loses core functions in about one‑fifth of runs, and that the first skill read determines the outcome, suggesting benchmarks should evaluate exclusive core functions and platforms should guard the initial read.

By Chaoliang Yan, Zihao Xu, Yuekang Li, Shangzhi Xu, Yi Liu, Gelei Deng, Siqi Ma
arXiv AI
3d ago

SkillContrast: Difference-Guided Text Selection for Agent Skill Reranking

SkillContrast is a training‑free selector that compares retrieved agent skills and keeps only the text that differs between them, preserving local context for a pretrained reranker. In experiments on 1,235 requests from SameCapRisk‑Bench, it improves clean‑hit rates by 54–72 over TF‑IDF query selection while using 51.1–58.8% fewer input tokens. The method shows that retaining differing text, rather than full skill bodies, is the key factor for better reranking performance.

By Jiandong Ding, Honglei Ji, Ming Liu, Tao Duan
arXiv AI
Sep 28

SkillFlow: Scalable and Efficient Agent Skill Retrieval System

SkillFlow is an open, multi-stage retrieval system that helps AI agents selectively load relevant skills from a large library of community-contributed SKILL.md definitions. The pipeline uses dense retrieval, two rounds of cross-encoder reranking, and LLM-based selection to balance recall and precision. Evaluations on SkillsBench and Terminal-Bench show that SkillFlow improves performance when high-quality skills are available, but retrieval alone does not help if the corpus lacks executable skills for the target domain.

By Fangzhou Li, Pagkratios Tagkopoulos, Ilias Tagkopoulos