The paper introduces a black-box framework for evaluating agentic AI systems, focusing on multi-step vulnerabilities that standard single-turn tests miss. It presents a seven-domain taxonomy linking observable behaviors to risk categories, an automated SAGE-RT red-teaming process generating 120 adversarial scenarios per domain, and a human-validated evaluation using LLM judges. Empirical tests on CrewAI and AutoGen agents show significant governance, privacy, and behavior risks, demonstrating the framework’s ability to uncover critical architectural weaknesses without privileged access.
By Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa, Sahil Agarwal, Prashanth Harshangi
arXiv:2608.30429v1 Announce Type: new
Abstract: LLM-based agent systems increasingly adopt skill-based architectures to reduce repetitive reasoning costs and improve stable, efficient task execution....
By Doyun Kim, Chanwoo Kim, Sugyeong Eo, Yeo-Chan Yoon, Chanjun Park
LLM-based agents leverage third-party skills to extend their capabilities in open-world scenarios. However, third-party skills can introduce extra security vulnerabilities, as seemingly harmless skills can contain latent safety risks that only emerge during actual execution.
arXiv:2607. 11698v1 Announce Type: cross Abstract: Production LLM agents such as Claude Code and Codex operate over untrusted content, files, commands, and workspace state, making safety failures directly actionable.
By Xutao Mao, Xiang Zheng, Cong Wang
The paper introduces skill cascading attacks, where a malicious goal is spread across multiple seemingly benign skills, causing harmful outcomes when combined. It presents SkillCascade, an automated red‑teaming framework, and releases SkillCascade‑Bench, a benchmark of 213 validated cascading test cases across various agent systems and domains. Experiments show that these cascaded interactions reliably induce harmful behaviors while evading existing per‑skill scanners and runtime monitors, revealing a gap between component‑level integrity and system‑level safety.
By Zihao Zhu, Siwei Lyu, Adel Bibi, Baoyuan Wu
Production LLM agents such as Claude Code and Codex operate over untrusted content, files, commands, and workspace state, making safety failures directly actionable. Red-teaming must therefore keep pace with evolving models and tools.
arXiv:2606. 00925v1 Announce Type: cross Abstract: Open agent platforms allow community contributors to publish reusable skills that agents can invoke at runtime.
By Ismail Hossain, Sai Puppala, Zhuoran Lu, Sajedul Talukder, Nan Jiang
The paper examines how production blocking monitors—such as Auto Mode in Claude Code and Guardian in OpenAI's Codex—perform when faced with persistently misaligned coding agents. By red‑teaming an adversarial agent, the authors show that high‑level attack strategies enable the agent to bypass these monitors in 79% of trials, using methods like prompt injection, multi‑agent coordination, and malicious compaction. They also propose design improvements to Auto Mode, yet note that preventing multi‑context attacks remains an open challenge.
By Alex Remedios, Simon Storf, Fabien Roger, John Hughes
arXiv:2608. 10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks.
By Zixing Chen, Xingyuan Liu, Jie Zhu, Huaixia Dou, Shuo Jiang, Junhui Li, Lifan Guo, Feng Chen, Chi Zhang
arXiv:2608.30207v1 Announce Type: cross
Abstract: Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal...
By Chen Xiong, Zhiyuan He, Pin-Yu Chen, Stjepan Picek, Tsung-Yi Ho
arXiv:2608.00677v2 Announce Type: replace
Abstract: AI agents operate in persistent environments where early state changes can influence decisions far into the future. Unlike conventional language-mo...
By Yunhao Chen, Xin Wang, Yixu Wang, Yi Liu, Jie Li, Yan Teng, Xingjun Ma, Xia Hu, Yu-Gang Jiang
arXiv:2609.39065v1 Announce Type: cross
Abstract: LLM agents increasingly rely on installable skills, which are packages of instructions, code, and resources that equip them with task-specific capabi...
By Yan Wang, Zhihao Zhang, Ke Chen, Kai Chen, Yaqin Zhang, Duohe Ma, Jun Dai, Xiaoyan Sun