arXiv AI

JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skills

arXiv:2608. 16465v1 Announce Type: new Abstract: Automated red-teaming has produced a growing collection of attack strategies, yet they typically remain scattered across prompts and workflows, making them difficult to systematically integrate, reuse, and improve at scale.

arXiv AI

Jun 9

PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits

arXiv:2510. 17947v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are improving at an exceptional rate.

By Neeladri Bhuiya, Madhav Aggarwal, Diptanshu Purwar

llms agents multimodal benchmarks safety

arXiv AI

Jun 11

Learning to Inject: Automated Prompt Injection via Reinforcement Learning

arXiv:2602. 05746v2 Announce Type: replace-cross Abstract: Prompt injection is a critical vulnerability in LLM agents, yet the strongest methods still rely on human red-teamers and hand-crafted prompts.

By Xin Chen, Jie Zhang, Florian Tram\`er

llms agents reinforcement-learning fine-tuning safety

arXiv Machine Learning

Aug 4

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.

By Ren-Jian Wang, Ke Xue, Zeyu Qin, Ziniu Li, Sheng Tang, Hao-Tian Li, Shengcai Liu, Zhi Yu, Yuanpeng Tan, Chao Qian

llms rag safety

arXiv AI

Jun 4

MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models

arXiv:2606. 04027v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising partially masked sequences under bidirectional context, exposing a safety surface distinct from autoregressive LLMs.

By Yingzi Ma, Zhengyue Zhao, Xiaogeng Liu, Minhui Xue, Yue Zhao, Chaowei Xiao

llms diffusion reinforcement-learning benchmarks safety

arXiv Machine Learning

Jul 30

GPT-Red: Automated Red Teaming via Self-Play at Scale

arXiv:2607. 26115v1 Announce Type: cross Abstract: We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs.

By Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal, Sam Toyer, Dylan Hunn, Stephanie Lin, Yuxin Wen, Xiangyu Qi, Christopher Wolff, Zizhao Wang, Milad Nasr, Sicheng Zhu, Chuan Guo, Juan Felipe Cer\'on Uribe, Kaiwen Wang, Aiden Low, Kai Xiao, Kai Chen

llms agents reinforcement-learning safety

arXiv AI

Jun 11

JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization

arXiv:2606. 11425v1 Announce Type: cross Abstract: Jailbreak attacks expose persistent safety weaknesses in large language models (LLMs), but existing stateless single-turn methods face a trade-off: hand-crafted prompts are expressive but static, while iterative prompt optimization can adapt but often relies on low-level mutations that require many target queries.

By Ge Shi, Jun Yin, Donglin Xie, Fangyi Liu, Yucan Li, Menglin Liu

llms reinforcement-learning multimodal safety

arXiv Machine Learning

Jul 24

PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses

arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.

By Chenlong Yin, Runpeng Geng, Yanting Wang, Jinyuan Jia

llms agents reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Jun 9

OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents

arXiv:2605. 08876v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly deployed as autonomous agents that execute tool-augmented, multi-step tasks, where latency is a critical factor for real-world applications.

By Xinyu Li, Ronghui Mu, Lin Li, Tianjin Huang, Gaojie Jin

llms agents safety

arXiv AI

Jun 2

Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs

arXiv:2603. 24511v2 Announce Type: replace-cross Abstract: We show that AI agents are capable of discovering novel algorithms for adversarial attacks against LLMs, advancing the state of the art on white-box jailbreaking and prompt injection evaluations.

By Alexander Panfilov, Peter Romov, Igor Shilov, Yves-Alexandre de Montjoye, Jonas Geiping, Maksym Andriushchenko

llms agents multimodal benchmarks safety

arXiv AI

Jul 20

Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking

arXiv:2602. 24009v4 Announce Type: replace-cross Abstract: Jailbreak techniques for large language models (LLMs) evolve faster than benchmarks, making robustness estimates stale and difficult to compare across papers due to drift in datasets, harnesses, and judging protocols.

By Zhicheng Fang, Jingjie Zheng, Chenxu Fu, Wei Xu

llms agents multimodal benchmarks safety

arXiv AI

Jun 16

Automated jailbreak attack targeting multiple defense strategies

arXiv:2606. 16751v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks.

By Qi Wang, Chengcheng Wan, Weijia He, Yanqing Li, Hanqi Sun, Xiaodong Gu, Jiangtao Wang

llms multimodal safety

arXiv Machine Learning

Jun 26

Jailbreaking for the Average Jane: Choosing Optimal Jailbreaks via Bandit Algorithms for Automatically Enhanced Queries

arXiv:2606. 26936v1 Announce Type: cross Abstract: With a profusion of jailbreaks for LLMs now widely known, a growing concern is that non-expert malicious actors ("the average Jane") could elicit actionable responses to malicious requests.

By Prarabdh Shukla, Ritik, Suhas Rao, Arpit Agarwal, Arjun Bhagoji

llms reinforcement-learning benchmarks safety

Related stories