arXiv AI By Jiandong Ding, Honglei Ji, Ming Liu, Tao Duan

Right Family, Wrong Skill: Benchmarking Risk Exposure in Agent Skill Retrieval

Read the original on arXiv AI →

arXiv:2606. 10388v2 Announce Type: replace-cross Abstract: Agent skill libraries are becoming routable software assets: a retrieved skill can contribute instructions, scripts, resource bindings, and execution assumptions to an agent.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
6d ago

SkillFlow: Scalable and Efficient Agent Skill Retrieval System

SkillFlow is an open, multi-stage retrieval system that helps AI agents selectively load relevant skills from a large library of community-contributed SKILL.md definitions. The pipeline uses dense retrieval, two rounds of cross-encoder reranking, and LLM-based selection to balance recall and precision. Evaluations on SkillsBench and Terminal-Bench show that SkillFlow improves performance when high-quality skills are available, but retrieval alone does not help if the corpus lacks executable skills for the target domain.

By Fangzhou Li, Pagkratios Tagkopoulos, Ilias Tagkopoulos
arXiv AI
Aug 19

TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation

TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It evaluates candidate skills against source evidence and nine safety properties, then tests them in a controlled environment to capture execution traces and identify failures. The system iteratively refines skills based on these results, achieving high precision in vulnerability detection and significantly improving task performance and security rates.

By Zhibo Zhang, Zhen Ouyang, Ling Shi, Kailong Wang
Hugging Face Trending Papers
Aug 18

TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation

TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It first checks functional claims against evidence and evaluates artifacts against nine safety properties, then tests admitted skills in a controlled environment to capture execution traces and identify failures. The approach achieves perfect precision and recall in vulnerability detection, significantly reduces attack success rates, and boosts task effectiveness and security rates in skill generation benchmarks.

arXiv AI
Aug 24

Evaluating Skills, Not Just Agents: Agentic Continuous Evaluation of Skills

The paper introduces ACES (Agentic Continuous Evaluation of Skills), a framework that evaluates reusable skills and capability packages by running paired live trials with and without a target skill, normalizing results into the Agent Trajectory Interchange Format (ATIF), and grading six runtime metrics to compute Skill Lift. ACES demonstrates that scan-only gates miss important aspects of skill performance, while the evaluation protocol reveals significant improvements in skill execution, behavior check, and skill efficiency across 145 real skills and 947 scored cases. The open‑source NVIDIA SkillEvaluator implementation enables reproducible, repository‑native assessment of agentic skills in production environments.

By Christopher Kevin, Narendran Raghavan, Jean-Francois Puget, Roshni Malani, Meghana Puvvadi, Moshe Abramovitch, Mohit Gupta, Rama Akkiraju, Subodh Prabhu, Yogesh Dangi, Wei Luo, Seong Hee Lee