Towards Data Science

Detecting Vulnerabilities in Agent Skills with SkillSpector: From Green Checkmark to Real Security Judgment

Static analysis nailed the malicious skill and over-flagged the useful one. The gap between those results is where human judgement actually earns its keep.

arXiv AI
Oct 31

Scan Before You Install: Detecting and Mitigating Local and Composite Vulnerabilities in Skills

The paper introduces a six‑module scanner for detecting unintended local and composite vulnerabilities in LLM‑based agent skills. Local Analysis achieved high recall and precision on benchmark datasets, outperforming existing scanners, while Composite Analysis accurately modeled inter‑skill dependencies and identified composite flaws. The authors validated findings with proof‑of‑concept attacks, disclosed cases to maintainers, and demonstrated effective mitigations that preserved legitimate functionality.

By Parth A. Gandhi, Avital Aviv, Prasanna N. Wudali, Yuval Elovici, Asaf Shabtai
arXiv AI
Oct 1

Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents

The paper titled "Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents" demonstrates how an attacker can bypass current skill‑scanning defenses by crafting malicious skills that evade both static analysis and LLM‑based semantic checks. By moving malicious payloads into natural language and distributing instructions across files, the white‑box attacker named Pretext achieves high evasion rates—up to 97% against a frozen detector and 77% against a co‑adaptive one—across three open‑source models.

By Tobias Kaisar, Aritra Dhar
arXiv AI
Oct 7

SkillPoison: Progressive Skill Poisoning via Successful Experiences

SkillPoison demonstrates that self‑improving large language model agents can be progressively poisoned through successful experiences without any single trajectory being malicious. The framework constructs successful experiences that reinforce a target behavior, then removes contextual constraints so the skill extractor generalizes to harmful behavior when misapplied. Experiments on three benchmarks show a 95.71% attack success rate while all injected experiences remain task‑correct and pass verification and lexical inspection.

By Lizhi Zhang, Xin He, Dianxuan Fu, Yuyuan Feng, Jiatong Li, Qi Wang, Xin Wang, Qinggang Zhang