arXiv AI
Jun 15
SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing
arXiv:2606. 14239v1 Announce Type: new Abstract: Agent skills are structured procedural packages that guide frozen LLM agents in specialized workflows.
Evaluating Skills, Not Just Agents: Agentic Continuous Evaluation of Skills
The paper introduces ACES (Agentic Continuous Evaluation of Skills), a framework that evaluates reusable skills and capability packages by running paired live trials with and without a target skill, normalizing results into the Agent Trajectory Interchange Format (ATIF), and grading six runtime metrics to compute Skill Lift. ACES demonstrates that scan-only gates miss important aspects of skill performance, while the evaluation protocol reveals significant improvements in skill execution, behavior check, and skill efficiency across 145 real skills and 947 scored cases. The open‑source NVIDIA SkillEvaluator implementation enables reproducible, repository‑native assessment of agentic skills in production environments.
Related stories
arXiv AI
Jun 2
When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
arXiv:2606. 00448v1 Announce Type: cross Abstract: LLM agents increasingly rely on community-contributed skills that expand an agent's operational capability set.
arXiv AI
Jun 17
A Framework for Evaluating Agentic Skills at Scale
arXiv:2606. 17819v1 Announce Type: cross Abstract: Agent skills -- structured, reusable knowledge artifacts that augment LLM agent capabilities -- have been rapidly adopted in industry, yet their cross-domain impact and use across commercial and open-source models remain under-studied, and no reusable methodology exists for evaluating an individual skill.
arXiv AI
Aug 19
TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation
TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It evaluates candidate skills against source evidence and nine safety properties, then tests them in a controlled environment to capture execution traces and identify failures. The system iteratively refines skills based on these results, achieving high precision in vulnerability detection and significantly improving task performance and security rates.
Hugging Face Trending Papers
Aug 18
TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation
TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It first checks functional claims against evidence and evaluates artifacts against nine safety properties, then tests admitted skills in a controlled environment to capture execution traces and identify failures. The approach achieves perfect precision and recall in vulnerability detection, significantly reduces attack success rates, and boosts task effectiveness and security rates in skill generation benchmarks.
arXiv AI
1d ago
ClawProBench: Trace-Aware Evaluation of AI Agents with Runtime Coverage and Frozen Workplace-Style Holdouts
arXiv:2608.22510v1 Announce Type: new Abstract: Agent benchmarks often evaluate only final answers even when agents run on stateful runtimes. We argue this under-specifies what is being evaluated: th...
arXiv AI
Aug 7
SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution
arXiv:2608. 05573v1 Announce Type: new Abstract: LLM agents increasingly execute long-horizon tasks through tool use and environment interaction, shifting evaluation from final-response scoring to verification of complete executions.
arXiv Computation and Language
1d ago
Signal or Noise? A Benchmark Study of Agent Skills in Web Development
arXiv:2608.23067v1 Announce Type: new Abstract: Agent Skills are reusable procedural modules that are increasingly injected into coding-agent sessions to encode framework conventions, anti-patterns,...
Hugging Face Trending Papers
Aug 6
SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution
LLM agents increasingly execute long-horizon tasks through tool use and environment interaction, shifting evaluation from final-response scoring to verification of complete executions. For skill-augmented agents, verification additionally requires the procedural knowledge encoded in task-time skills, because this knowledge indicates what evidence to inspect and which failures are task-critical.
arXiv AI
Jul 29
HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following
arXiv:2607. 25398v1 Announce Type: new Abstract: Language-model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to let it govern every action that follows.
arXiv AI
Aug 11
SkillSentry: Reliable Skill Execution for LLM Agents via Runtime Assurance
arXiv:2608. 09253v1 Announce Type: new Abstract: LLM agents are increasingly equipped with skills to perform complex tasks through multi-step reasoning and tool use.
arXiv AI
Jun 2
SkillSmith: Co-Evolving Skills and Tools for Self-Improving Agent Systems
arXiv:2606. 01314v1 Announce Type: new Abstract: Recent self-evolving agents have shown that skills can be discovered, refined, and accumulated through execution.