arXiv AI August 28, 2026 By Junjie Zhang, Hui Liu, Kecheng Chen, Xianbo Mo, Changsheng Chen, Haoliang Li

RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution

RedEvoAgent is a black-box red‑teaming agent that transforms cross‑case attack trajectories into concise, human‑readable attack skills. It evolves these skills by profiling tool effectiveness, attributing tool credit, and applying a validation ratchet to keep only improvements. Experiments demonstrate that RedEvoAgent outperforms fixed and agentic baselines, enhances tool efficiency, and transfers across attacker models and target execution harnesses.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 18

JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skills

arXiv:2608. 16465v1 Announce Type: new Abstract: Automated red-teaming has produced a growing collection of attack strategies, yet they typically remain scattered across prompts and workflows, making them difficult to systematically integrate, reuse, and improve at scale.

By Xiaoyu Wen, Jiajia Li, Zhida He, Peng Yu, Chenxu Wang, Han Qi, Ziyuan Zhou, Cheng Jin, Ying Wen, Xingcheng Xu, Shuyue Hu, Tianhang Zheng, Chaochao Lu, Qiaosheng Zhang

llms agents multimodal safety

arXiv Machine Learning

Jun 9

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

arXiv:2602. 16346v4 Announce Type: replace-cross Abstract: LLM-based agents execute real-world workflows via tools and memory.

By Nivya Talokar, Ayush K Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut

llms agents benchmarks safety

arXiv AI

Sep 1

T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search

The paper introduces T-MAP, a trajectory‑aware evolutionary search technique designed to red‑team large language model agents by exploiting vulnerabilities that arise during multi‑step tool execution. Unlike traditional methods that focus on harmful text, T‑MAP uses execution trajectories to generate adversarial prompts that bypass safety guardrails and achieve harmful objectives through actual tool interactions. Experiments across various Model Context Protocol environments show that T‑MAP outperforms baseline methods in attack realization rate and remains effective against advanced models such as GPT‑5.2, Gemini‑3‑Pro, Qwen3.5, and GLM‑5.

By Hyomin Lee, Sangwoo Park, Yumin Choi, Sohyun An, Seanie Lee, Sung Ju Hwang

llms agents safety

arXiv AI

Jun 9

PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits

arXiv:2510. 17947v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are improving at an exceptional rate.

By Neeladri Bhuiya, Madhav Aggarwal, Diptanshu Purwar

llms agents multimodal benchmarks safety

arXiv AI

Jun 16

Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClaw

arXiv:2605. 11047v2 Announce Type: replace-cross Abstract: Agentic language-model systems increasingly rely on mutable execution contexts, including files, memory, tools, skills, and auxiliary artifacts, creating security risks beyond explicit user prompts.

By Hongwei Yao, Yiming Liu, Yiling He, Bingrun Yang

agents robotics benchmarks safety

arXiv AI

Aug 12

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

arXiv:2608. 10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks.

By Zixing Chen, Xingyuan Liu, Jie Zhu, Huaixia Dou, Shuo Jiang, Junhui Li, Lifan Guo, Feng Chen, Chi Zhang

llms agents multimodal benchmarks safety

One email a morning, machine-written

Related stories

JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skills

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search

PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits

Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClaw

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems