AlcaTRAz is a prompt‑level defense that uses rule trees to insert controlled character‑level perturbations into input text, disrupting jailbreak attacks without modifying or retraining the target LLM. It operates solely on the input, making it suitable for black‑box deployments, and was evaluated on 33 open‑weight models and 22 jailbreak types, outperforming three baseline defenses in 73.4 % of model‑attack combinations. While it significantly reduces high‑severity jailbreak success, it does not eliminate it and is intended as one layer of a broader defense strategy.
arXiv:2607. 19424v1 Announce Type: cross Abstract: The assessment of jailbreak attacks against large language models currently suffers from inconsistent evaluation criteria and methods, leading to unreliable estimates of attack success rates.
The paper investigates safety risks in model merging, showing that even when all constituent models are individually safety‑aligned, merging can expose a jailbreak vulnerability rooted in the pretrained foundation model. It introduces Basin‑Aware Jailbreak (BAJ), a min–max optimization method that generates adversarial suffixes transferable across merged models sharing the same backbone, without needing the exact merging coefficients or checkpoints. Experiments demonstrate BAJ’s high transfer success rates across diverse backbones and merging settings, and its resilience against existing defenses.
The paper presents a systematic study of combining defenses against jailbreak attacks on Large Language Models across different pipeline stages. It introduces a standardized evaluation framework that defines attack-success-rate, controls query budgets, and applies explicit fairness rules. Across 19 attacks and 15 defenses, the study finds that no single defense dominates, but carefully selected combinations can provide strong safety with minimal loss of utility, offering practical guidance for layered defense pipelines.
MLCommons Jailbreak Benchmark v1.0 is a new methodology for assessing how well large language models resist single‑turn, text‑based jailbreak attacks. It evaluates eight open‑weight systems with 264 seed prompts across eleven hazard categories, using the AILuminate Assessment Standard v1.4 to measure the Resilience Gap—the difference in safety performance between baseline and adversarial conditions. The benchmark found that unsafe‑response rates rose from 11.08% to 18.65% under jailbreak conditions, yielding an average Resilience Gap of 7.57% and highlighting variability in attack effectiveness and evaluator reliability.
arXiv:2606. 03486v1 Announce Type: cross Abstract: Large language models remain vulnerable to jailbreak attacks that hide harmful intent behind seemingly ordinary requests such as role-play, translation, encoding, adversarial suffixes, and multi-turn buildup.
The paper "Jailbreaking in the Haystack" introduces NINJA, a jailbreak technique that exploits long-context language models by appending benign, model-generated content to harmful user goals. It demonstrates that the position of harmful goals within the context is crucial for safety, and shows that NINJA significantly boosts attack success rates on models such as LLaMA, Qwen, Mistral, and Gemini. Unlike previous methods, NINJA is low-resource, transferable, less detectable, and compute‑optimal, revealing that carefully crafted benign long contexts can expose fundamental vulnerabilities in modern LMs.
arXiv:2506. 22666v3 Announce Type: replace-cross Abstract: The rise of API-only access to state-of-the-art LLMs highlights the need for effective black-box jailbreak methods to identify model vulnerabilities in real-world settings.
arXiv:2508. 10029v3 Announce Type: replace-cross Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations.
arXiv:2609.26185v1 Announce Type: cross Abstract: Large Language Models (LLMs) demonstrate impressive capabilities across many applications but remain vulnerable to jailbreak attacks, which elicit ha...
arXiv:2606. 11425v1 Announce Type: cross Abstract: Jailbreak attacks expose persistent safety weaknesses in large language models (LLMs), but existing stateless single-turn methods face a trade-off: hand-crafted prompts are expressive but static, while iterative prompt optimization can adapt but often relies on low-level mutations that require many target queries.
arXiv:2610.15207v1 Announce Type: cross Abstract: Large Language Models (LLMs) have been shown to be computationally expensive and vulnerable to jailbreak attacks, in which adversarial prompts bypass...
arXiv:2608. 03070v1 Announce Type: cross Abstract: Frontier AI model developers increasingly rely on layered safeguards to prevent catastrophic misuse, but little public evidence exists on how much protection these safeguards provide, or how consistently across developers.