OpenAI Blog
Jul 15
GPT-Red: Unlocking Self-Improvement for Robustness
Explore GPT-Red, OpenAI’s automated red teaming system that uses self-play to improve AI safety, alignment, and prompt injection robustness.
GPT-Red: Automated Red Teaming via Self-Play at Scale
arXiv:2607. 26115v1 Announce Type: cross Abstract: We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs.
Related stories
arXiv Machine Learning
Jul 24
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.
arXiv AI
Jun 9
Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
arXiv:2606. 09701v1 Announce Type: cross Abstract: AI red teaming must continually adapt to evolving attackers and defenders.
arXiv Machine Learning
Jul 1
Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance
arXiv:2605. 00553v3 Announce Type: replace Abstract: Large Language Model (LLM) Red-Teaming, which proactively identifies vulnerabilities of LLMs, is an essential process for ensuring safety.
arXiv Machine Learning
Aug 4
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.
arXiv AI
Aug 6
Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)
arXiv:2608. 04317v1 Announce Type: cross Abstract: Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated almost exclusively against static, heuristic red agents, leaving their robustness against adaptive threats critically understudied.
arXiv AI
1d ago
JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skills
arXiv:2608. 16465v1 Announce Type: new Abstract: Automated red-teaming has produced a growing collection of attack strategies, yet they typically remain scattered across prompts and workflows, making them difficult to systematically integrate, reuse, and improve at scale.
arXiv AI
Aug 12
REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
arXiv:2608. 10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks.
arXiv AI
Jul 7
NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms
arXiv:2606. 20408v3 Announce Type: replace-cross Abstract: Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized.
arXiv Machine Learning
Jul 7
Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
arXiv:2506. 07468v4 Announce Type: replace Abstract: Conventional large language model (LLM) safety alignment relies on a reactive, disjoint loop: attackers exploit a static model, then defenders patch exposed vulnerabilities.
arXiv AI
Jun 19
LLM agent safety, multi-turn red-teaming, jailbreak benchmarks, adversarial robustness, safety-critical systems
arXiv:2606. 20408v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized.
arXiv Machine Learning
Jun 9
OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents
arXiv:2605. 08876v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly deployed as autonomous agents that execute tool-augmented, multi-step tasks, where latency is a critical factor for real-world applications.