arXiv AI June 9, 2026 By Blake Bullwinkel, Eugenia Kim, Amanda Minnich, Mark Russinovich

Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO

arXiv:2606. 09701v1 Announce Type: cross Abstract: AI red teaming must continually adapt to evolving attackers and defenders.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 24

PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses

arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.

By Chenlong Yin, Runpeng Geng, Yanting Wang, Jinyuan Jia

llms agents reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Aug 4

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.

By Ren-Jian Wang, Ke Xue, Zeyu Qin, Ziniu Li, Sheng Tang, Hao-Tian Li, Shengcai Liu, Zhi Yu, Yuanpeng Tan, Chao Qian

llms rag safety

arXiv AI

Jun 2

Safety Alignment of LMs via Non-cooperative Games

arXiv:2512. 20806v3 Announce Type: replace Abstract: Ensuring the safety of language models (LMs) while maintaining their usefulness remains a critical challenge in AI alignment.

By Anselm Paulus, Ilia Kulikov, Brandon Amos, R\'emi Munos, Ivan Evtimov, Kamalika Chaudhuri, Arman Zharmagambetov

llms agents reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 30

GPT-Red: Automated Red Teaming via Self-Play at Scale

arXiv:2607. 26115v1 Announce Type: cross Abstract: We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs.

By Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal, Sam Toyer, Dylan Hunn, Stephanie Lin, Yuxin Wen, Xiangyu Qi, Christopher Wolff, Zizhao Wang, Milad Nasr, Sicheng Zhu, Chuan Guo, Juan Felipe Cer\'on Uribe, Kaiwen Wang, Aiden Low, Kai Xiao, Kai Chen

llms agents reinforcement-learning safety

arXiv Machine Learning

Jul 1

Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance

arXiv:2605. 00553v3 Announce Type: replace Abstract: Large Language Model (LLM) Red-Teaming, which proactively identifies vulnerabilities of LLMs, is an essential process for ensuring safety.

By Minchan Kwon, Sunghyun Baek, Minseo Kim, Jaemyung Yu, Dongyoon Han, Junmo Kim

llms safety

arXiv AI

Aug 6

Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)

arXiv:2608. 04317v1 Announce Type: cross Abstract: Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated almost exclusively against static, heuristic red agents, leaving their robustness against adaptive threats critically understudied.

By Ryozo Masukawa, Ian Bryant, Armita Kazeminajafabadi, Sanggeon Yun, Hyunwoo Oh, SungHeon Jeong, Nathaniel D. Bastian, Mahdi Imani, Mohsen Imani

llms agents reinforcement-learning benchmarks

Related stories

PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

Safety Alignment of LMs via Non-cooperative Games

GPT-Red: Automated Red Teaming via Self-Play at Scale

Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance

Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)