OpenAI Blog
Jul 15
GPT-Red: Unlocking Self-Improvement for Robustness
Explore GPT-Red, OpenAI’s automated red teaming system that uses self-play to improve AI safety, alignment, and prompt injection robustness.
Advancing red teaming with people and AI
Advancing red teaming with people and AI
Related stories
arXiv Machine Learning
Jul 30
GPT-Red: Automated Red Teaming via Self-Play at Scale
arXiv:2607. 26115v1 Announce Type: cross Abstract: We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs.
arXiv Machine Learning
Jun 2
NestRL: A Nested Training Regime for Mutual Adaptation in Human-AI Teaming
arXiv:2602. 17737v2 Announce Type: replace-cross Abstract: Mutual adaptation is a central challenge in human-AI teaming, as humans naturally adjust their strategies in response to an AI agent's behavior.
arXiv AI
Jul 3
What Types of Human-AI Teams Exist?
arXiv:2607. 02198v1 Announce Type: cross Abstract: Human-AI teaming has received increasing attention in the literature.
arXiv AI
Jun 10
Human-AI Teaming Through the Lens of Calibration
arXiv:2606. 10906v1 Announce Type: cross Abstract: We study models for human-AI teaming through the lens of statistical calibration.
arXiv AI
2d ago
AI Evaluation Should Work With Humans
arXiv:2608. 13577v1 Announce Type: new Abstract: This position paper argues that the dominant paradigm of AI evaluation (which focuses on superhuman autonomous performance and so implicitly targets the goal of replacing humans) is guiding AI development in the wrong direction.
OpenAI Blog
Sep 19, 2023
OpenAI Red Teaming Network
We’re announcing an open call for the OpenAI Red Teaming Network and invite domain experts interested in improving the safety of OpenAI’s models to join our efforts.
arXiv AI
Jun 18
Searching for Synergy in Shared Workspace Human-AI Collaboration
arXiv:2606. 18413v1 Announce Type: new Abstract: Automated AI agents are increasingly capable, yet many scientific and professional tasks require human judgment and contextual expertise.
arXiv AI
Jul 9
Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety
arXiv:2607. 07695v1 Announce Type: new Abstract: We introduce institutional red-teaming, an evaluation methodology for testing deployment rules in multi-agent AI: hold the agents, objectives, and task state fixed, vary only one rule, and attribute the resulting change in collective behavior to that rule.
arXiv AI
Aug 5
Mixed-Initiative Human-Robot Teaming under Suboptimality with Online Bayesian Adaptation
arXiv:2403. 16178v2 Announce Type: replace-cross Abstract: For effective human-agent teaming, robots and other artificial intelligence (AI) agents must infer their human partner's abilities and behavioral response patterns and adapt accordingly.
arXiv AI
Jun 9
Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
arXiv:2606. 09701v1 Announce Type: cross Abstract: AI red teaming must continually adapt to evolving attackers and defenders.
arXiv AI
Jun 17
Algorithmic Prompt Generation for Diverse Human-like Teaming and Communication with Large Language Models
arXiv:2504. 03991v2 Announce Type: replace-cross Abstract: Understanding how humans collaborate and communicate in teams is essential for improving human-agent teaming and AI-assisted decision-making.