CoFiBreak introduces a coarse‑to‑fine diversity reward for Group Relative Policy Optimization (GRPO) that enhances the variety of attack strategies in automated red‑teaming of large language models while maintaining high attack success and low inference cost. The coarse reward encourages underrepresented strategies within each GRPO group, and the fine reward discourages reuse of personas, framings, and output formats after harmful‑intent information is removed from prompt embeddings. Trained against Llama3.1‑8B‑Instruct, CoFiBreak achieves 1.6–2.0× more distinct successful strategies than the strongest baseline, attains 98.8% ASR@10 on AdvBench, and transfers effectively to unseen frontier LLMs such as Claude‑Opus‑4.6.
arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.
The paper introduces CoRL, a co-evolutionary reinforcement learning framework designed to defend against adaptive indirect prompt-injection attacks on tool-augmented language agents. CoRL operates in three stages—attacker fine‑tuning, bilateral Co‑PPO training, and defender fine‑tuning—using verifier‑grounded repairs to adapt to changing attack strategies. Experiments on 1,514 executions show that CoRL reduces attack success rates to 0% while improving task utility, demonstrating its effectiveness against adaptive adversaries.
arXiv:2603. 13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents.
arXiv:2610.15513v1 Announce Type: cross Abstract: Safety-aligned large language models (LLMs) remain vulnerable to adversarial prompts that bypass their safety training. Existing automated red-teamin...
arXiv:2606. 09701v1 Announce Type: cross Abstract: AI red teaming must continually adapt to evolving attackers and defenders.
arXiv:2606. 15441v1 Announce Type: cross Abstract: Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.
EvoFlint is an evolutionary atlas that maps multi‑turn vulnerabilities in large language models by treating attack discovery as a search problem rather than a generation task. It uses evolutionary quality‑diversity search to evolve phased conversation plans, employing Pareto fitness for success rate and severity, novelty search for diversity, and a generation‑level memory to incorporate model insights. The resulting risk‑indexed archive, tested on HarmBench, shows high attack success rates across models such as Claude Sonnet, GPT‑5, and Qwen3, revealing which harm categories each model’s safety training covers or misses.
arXiv:2609.21208v1 Announce Type: new Abstract: Self-play methods that co-train a single language model as both coder and test author promise to move code-generation RL beyond fixed test suites, but...
The paper introduces Meta-Adaptive Multimodal Jailbreaking (MAMJ), a method that jointly optimizes an attack strategy prompt and attacker weights to generate more effective jailbreaks against vision‑language models. Using an LLM‑based critique to refine the strategy and group‑level success‑rate rewards to update the weights, MAMJ achieves high attack success rates on MM‑SafetyBench, outperforming existing baselines by up to 24.1 percentage points. The learned attacker also transfers to unseen models and remains robust against typical defenses, highlighting a systemic vulnerability in current VLMs.
arXiv:2610.35743v1 Announce Type: new Abstract: Existing jailbreak attackers fall into two classes: static planners fix their single- or multi-turn adversarial plans in advance, remaining simple but...
Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense.
arXiv:2508. 10029v3 Announce Type: replace-cross Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations.