OpenAI Blog
Jul 17, 2025
ChatGPT agent System Card
ChatGPT agent System Card: OpenAI’s agentic model unites research, browser automation, and code tools with safeguards under the Preparedness Framework.
Continuously hardening ChatGPT Atlas against prompt injection
OpenAI is strengthening ChatGPT Atlas against prompt injection attacks using automated red teaming trained with reinforcement learning. This proactive discover-and-patch loop helps identify novel exploits early and harden the browser agent’s defenses as AI becomes more agentic.
Related stories
OpenAI Blog
Jul 15
GPT-Red: Unlocking Self-Improvement for Robustness
Explore GPT-Red, OpenAI’s automated red teaming system that uses self-play to improve AI safety, alignment, and prompt injection robustness.
OpenAI Blog
Nov 7, 2025
Understanding prompt injections: a frontier security challenge
Prompt injections are a frontier security challenge for AI systems. Learn how these attacks work and how OpenAI is advancing research, training models, and building safeguards for users.
arXiv AI
Jun 12
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
arXiv:2606. 12737v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly evolving into agentic systems that interact with external tools and environments, introducing new security risks such as indirect prompt injection attacks through untrusted external sources.
arXiv Machine Learning
Jul 24
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.
OpenAI Blog
Mar 11
Designing AI agents to resist prompt injection
How ChatGPT defends against prompt injection and social engineering by constraining risky actions and protecting sensitive data in agent workflows.
arXiv AI
Jun 24
RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems
arXiv:2606. 23927v1 Announce Type: new Abstract: Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities.
arXiv AI
Jul 23
Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents
arXiv:2607. 19837v1 Announce Type: new Abstract: Traditional pentesting uses reconnaissance at each step to uncover unseen weaknesses, build stronger attacks, and advance the objective; we argue that AI agents require the same treatment.
arXiv AI
Sep 1
T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search
The paper introduces T-MAP, a trajectory‑aware evolutionary search technique designed to red‑team large language model agents by exploiting vulnerabilities that arise during multi‑step tool execution. Unlike traditional methods that focus on harmful text, T‑MAP uses execution trajectories to generate adversarial prompts that bypass safety guardrails and achieve harmful objectives through actual tool interactions. Experiments across various Model Context Protocol environments show that T‑MAP outperforms baseline methods in attack realization rate and remains effective against advanced models such as GPT‑5.2, Gemini‑3‑Pro, Qwen3.5, and GLM‑5.
arXiv AI
Jul 17
Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems
arXiv:2607. 14611v1 Announce Type: cross Abstract: A growing class of agentic systems maintain persistent state across sessions through memory files, behavioral preferences, and knowledge bases.
OpenAI Blog
Aug 17
The Defender’s Window
AI is reshaping cybersecurity for attackers and defenders alike. Learn how OpenAI is strengthening its defenses and what security teams can do now.
arXiv AI
Jun 10
Assessing Automated Prompt Injection Attacks in Agentic Environments
arXiv:2606. 10525v1 Announce Type: cross Abstract: Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain underexplored in realistic agentic settings.