The paper introduces T-MAP, a trajectory‑aware evolutionary search technique designed to red‑team large language model agents by exploiting vulnerabilities that arise during multi‑step tool execution. Unlike traditional methods that focus on harmful text, T‑MAP uses execution trajectories to generate adversarial prompts that bypass safety guardrails and achieve harmful objectives through actual tool interactions. Experiments across various Model Context Protocol environments show that T‑MAP outperforms baseline methods in attack realization rate and remains effective against advanced models such as GPT‑5.2, Gemini‑3‑Pro, Qwen3.5, and GLM‑5.
arXiv:2405.18540v3 Announce Type: replace-cross Abstract: Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of larg...
arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.
arXiv:2606. 23927v1 Announce Type: new Abstract: Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities.
RedEvoAgent is a black-box red‑teaming agent that transforms cross‑case attack trajectories into concise, human‑readable attack skills. It evolves these skills by profiling tool effectiveness, attributing tool credit, and applying a validation ratchet to keep only improvements. Experiments demonstrate that RedEvoAgent outperforms fixed and agentic baselines, enhances tool efficiency, and transfers across attacker models and target execution harnesses.
The paper introduces a black-box framework for evaluating agentic AI systems, focusing on multi-step vulnerabilities that standard single-turn tests miss. It presents a seven-domain taxonomy linking observable behaviors to risk categories, an automated SAGE-RT red-teaming process generating 120 adversarial scenarios per domain, and a human-validated evaluation using LLM judges. Empirical tests on CrewAI and AutoGen agents show significant governance, privacy, and behavior risks, demonstrating the framework’s ability to uncover critical architectural weaknesses without privileged access.
arXiv:2605. 11047v2 Announce Type: replace-cross Abstract: Agentic language-model systems increasingly rely on mutable execution contexts, including files, memory, tools, skills, and auxiliary artifacts, creating security risks beyond explicit user prompts.
arXiv:2607. 11698v1 Announce Type: cross Abstract: Production LLM agents such as Claude Code and Codex operate over untrusted content, files, commands, and workspace state, making safety failures directly actionable.
arXiv:2505. 14289v2 Announce Type: replace Abstract: Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) are increasingly deployed yet vulnerable to Environmental Injection Attacks (EIAs).
arXiv:2606. 00801v1 Announce Type: cross Abstract: Current approaches to LLM adversarial testing suffer from coverage gaps: manual red-teaming does not scale, LLM-as-attacker methods exhibit mode collapse, and gradient-based approaches produce uninterpretable gibberish.
EvoFlint is an evolutionary atlas that maps multi‑turn vulnerabilities in large language models by treating attack discovery as a search problem rather than a generation task. It uses evolutionary quality‑diversity search to evolve phased conversation plans, employing Pareto fitness for success rate and severity, novelty search for diversity, and a generation‑level memory to incorporate model insights. The resulting risk‑indexed archive, tested on HarmBench, shows high attack success rates across models such as Claude Sonnet, GPT‑5, and Qwen3, revealing which harm categories each model’s safety training covers or misses.
Production LLM agents such as Claude Code and Codex operate over untrusted content, files, commands, and workspace state, making safety failures directly actionable. Red-teaming must therefore keep pace with evolving models and tools.
arXiv:2606. 07992v1 Announce Type: new Abstract: As the Model Context Protocol (MCP) standardizes tool-calling for autonomous agents, it introduces a critical, unexamined attack surface: the error-handling loop.