OpenAI Blog
Jul 17, 2025
ChatGPT agent System Card
ChatGPT agent System Card: OpenAI’s agentic model unites research, browser automation, and code tools with safeguards under the Preparedness Framework.
Continuously hardening ChatGPT Atlas against prompt injection
OpenAI is strengthening ChatGPT Atlas against prompt injection attacks using automated red teaming trained with reinforcement learning. This proactive discover-and-patch loop helps identify novel exploits early and harden the browser agent’s defenses as AI becomes more agentic.
Related stories
OpenAI Blog
Jul 15
GPT-Red: Unlocking Self-Improvement for Robustness
Explore GPT-Red, OpenAI’s automated red teaming system that uses self-play to improve AI safety, alignment, and prompt injection robustness.
OpenAI Blog
Nov 7, 2025
Understanding prompt injections: a frontier security challenge
Prompt injections are a frontier security challenge for AI systems. Learn how these attacks work and how OpenAI is advancing research, training models, and building safeguards for users.
arXiv AI
Jun 12
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
arXiv:2606. 12737v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly evolving into agentic systems that interact with external tools and environments, introducing new security risks such as indirect prompt injection attacks through untrusted external sources.
arXiv Machine Learning
Jul 24
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.
OpenAI Blog
Mar 11
Designing AI agents to resist prompt injection
How ChatGPT defends against prompt injection and social engineering by constraining risky actions and protecting sensitive data in agent workflows.
arXiv AI
Jun 24
RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems
arXiv:2606. 23927v1 Announce Type: new Abstract: Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities.
arXiv AI
Jul 23
Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents
arXiv:2607. 19837v1 Announce Type: new Abstract: Traditional pentesting uses reconnaissance at each step to uncover unseen weaknesses, build stronger attacks, and advance the objective; we argue that AI agents require the same treatment.
arXiv AI
Jul 17
Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems
arXiv:2607. 14611v1 Announce Type: cross Abstract: A growing class of agentic systems maintain persistent state across sessions through memory files, behavioral preferences, and knowledge bases.
OpenAI Blog
2d ago
The Defender’s Window
AI is reshaping cybersecurity for attackers and defenders alike. Learn how OpenAI is strengthening its defenses and what security teams can do now.
arXiv AI
Jun 10
Assessing Automated Prompt Injection Attacks in Agentic Environments
arXiv:2606. 10525v1 Announce Type: cross Abstract: Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain underexplored in realistic agentic settings.
arXiv AI
Jun 2
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations
arXiv:2606. 02240v1 Announce Type: cross Abstract: Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither writes nor controls.