arXiv Computation and Language

OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution

arXiv AI
Sep 1

T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search

The paper introduces T-MAP, a trajectory‑aware evolutionary search technique designed to red‑team large language model agents by exploiting vulnerabilities that arise during multi‑step tool execution. Unlike traditional methods that focus on harmful text, T‑MAP uses execution trajectories to generate adversarial prompts that bypass safety guardrails and achieve harmful objectives through actual tool interactions. Experiments across various Model Context Protocol environments show that T‑MAP outperforms baseline methods in attack realization rate and remains effective against advanced models such as GPT‑5.2, Gemini‑3‑Pro, Qwen3.5, and GLM‑5.

By Hyomin Lee, Sangwoo Park, Yumin Choi, Sohyun An, Seanie Lee, Sung Ju Hwang
arXiv AI
Jun 24

RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems

arXiv:2606. 23927v1 Announce Type: new Abstract: Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities.

By Yarin Yerushalmi Levi, Roy Betser, Amit Giloni, Lidor Erez, Itay Gershon, Oren Rachmil, Sindhu Padakandla, Roman Vainshtein
arXiv AI
Sep 11

Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery

The paper introduces a black-box framework for evaluating agentic AI systems, focusing on multi-step vulnerabilities that standard single-turn tests miss. It presents a seven-domain taxonomy linking observable behaviors to risk categories, an automated SAGE-RT red-teaming process generating 120 adversarial scenarios per domain, and a human-validated evaluation using LLM judges. Empirical tests on CrewAI and AutoGen agents show significant governance, privacy, and behavior risks, demonstrating the framework’s ability to uncover critical architectural weaknesses without privileged access.

By Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa, Sahil Agarwal, Prashanth Harshangi
arXiv AI
Aug 11

SHE: Trajectory-driven Safety Harness Evolution for LLM Agents

arXiv:2608. 09885v1 Announce Type: new Abstract: The safety of large language model (LLM) agents depends not only on model weights but also on the agent harness that manages context, memory, tools, permissions, and runtime control.

By Wanying Qu, Qinghua Mao, Yu Li, Jiyao Liu, Xin Zhang, Dadi Guo, Yanxu Zhu, Qingyu Liu, Leitao Yuan, Xi Lin, Shanfeng Zhu, Yanwei Fu, Jing Shao, Xia Hu, Dongrui Liu