arXiv AI
Jun 9
Self-Mined Hardness for Safety Fine-Tuning
arXiv:2605. 03226v2 Announce Type: replace-cross Abstract: Safety fine-tuning of language models typically requires a curated adversarial dataset.
Identically Trained Language Models Break on Different Jailbreaks
The study evaluates how identically trained language models respond to jailbreak prompts, finding that models trained with the same pretraining run exhibit highly correlated non-refusal rates (average 0.984), while different pretraining seeds reduce correlation to 0.811 and different data recipes slightly lower it further to 0.779. This indicates that a model’s susceptibility to jailbreaks is largely determined by its specific pretraining run rather than the data used. Additionally, each pretraining run leaves a distinct fingerprint, enabling identification of the run from a model’s responses to a set of prompts.
Related stories
arXiv Machine Learning
Jun 16
Rapid Poison: Practical Poisoning Attacks Against the Rapid Response Framework
arXiv:2606. 16242v1 Announce Type: new Abstract: The Rapid Response (RR) framework, deployed in production systems, including Anthropic's ASL-3 safeguards, continuously improves jailbreak-detection classifiers.
arXiv AI
Aug 26
ADVERSA: Measuring Multi-Turn Guardrail Degradation and Judge Reliability in Large Language Models
The paper introduces ADVERSA, an automated red‑teaming framework that evaluates large language model safety over multiple turns by tracking continuous compliance trajectories instead of binary jailbreak outcomes. Using a fine‑tuned 70B attacker model and a structured 5‑point rubric, the authors conduct controlled experiments on three frontier victim models, measuring guardrail degradation and judge reliability through a triple‑judge consensus. Results show a 26.7% jailbreak rate with most breaches occurring early, and the study documents inter‑judge agreement, attacker drift, and attacker refusals as key factors affecting safety assessment.
arXiv Machine Learning
Jul 10
Before the Last Token: Diagnosing Final-Token Safety Probe Failures
arXiv:2605. 12726v2 Announce Type: replace Abstract: Final-token safety probes monitor a single hidden state after prompt prefill, but jailbreak prompts can contain probe-visible unsafe evidence distributed across earlier user-token representations that is missed by this readout.
arXiv Computation and Language
Oct 23
Beyond Attack Success Rate: Cost-Aware Metric for Evaluating Iterative LLM Jailbreaks
The paper introduces a new metric, ASR_STEP, that extends the traditional attack success rate (ASR) by penalizing repeated calls to a victim large language model during jailbreak attempts. Experiments on JailBreakBench show that GCG’s scores drop by 7.0–8.7 percentage points after discounting, while PAP’s scores remain unchanged, and PAIR’s discount ranges from 3.4–12 percentage points depending on the iterative strategy. The authors argue that reporting both ASR and ASR_STEP makes the cost of repeated victim-model interactions explicit.
arXiv AI
Sep 10
The Geometry of Refusal: Why Post-Hoc Safety Is Fragile and Pretraining-Time Safety Persists
arXiv:2609.06934v1 Announce Type: cross Abstract: Post-hoc safety training (RLHF, DPO) is the dominant way to align large language models, yet jailbreaks (Zou et al., 2023b), fine-tuning attacks (Qi...
arXiv AI
Aug 10
Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models
arXiv:2605. 00123v3 Announce Type: replace Abstract: Safety trained large language models (LLMs) can often be induced to answer harmful requests through jailbreak prompts.
arXiv AI
Sep 28
Depth, Not Breadth: Best-of-N Jailbreaking Beyond Surface Noise
The paper investigates how allocating a query budget to structural depth rather than surface variation improves jailbreak success against the SAGE self‑check defense. By using a best‑of‑N approach over a code‑completion encoding, the authors achieve 67%, 22%, and 15% success rates on three open‑weight targets—far exceeding the 4.7% and 3.0% rates of single‑draw encoding and character‑search methods. The study demonstrates that depth of encoding and breadth of variation independently undermine transform and gate defenses, and that repeated sampling can inflate perceived robustness.
arXiv Machine Learning
Aug 28
A Single Suffix to Break Them All: Basin-Aware Jailbreaks for Merged Model Families
The paper investigates safety risks in model merging, showing that even when all constituent models are individually safety‑aligned, merging can expose a jailbreak vulnerability rooted in the pretrained foundation model. It introduces Basin‑Aware Jailbreak (BAJ), a min–max optimization method that generates adversarial suffixes transferable across merged models sharing the same backbone, without needing the exact merging coefficients or checkpoints. Experiments demonstrate BAJ’s high transfer success rates across diverse backbones and merging settings, and its resilience against existing defenses.
arXiv Computation and Language
Sep 1
The Fragility of Jailbreak Robustness Across Operational States
The study shows that jailbreak robustness in language models is highly sensitive to operational-state changes. Even minor alterations to system prompts, not intended to affect safety, can dramatically shift attack success rates across seven aligned models and three jailbreak methods. The authors link these variations to changes in hidden representations along a refusal-related axis, which can predict jailbreak outcomes.
arXiv AI
Oct 5
MLCommons Jailbreak Benchmark v1.0
MLCommons Jailbreak Benchmark v1.0 is a new methodology for assessing how well large language models resist single‑turn, text‑based jailbreak attacks. It evaluates eight open‑weight systems with 264 seed prompts across eleven hazard categories, using the AILuminate Assessment Standard v1.4 to measure the Resilience Gap—the difference in safety performance between baseline and adversarial conditions. The benchmark found that unsafe‑response rates rose from 11.08% to 18.65% under jailbreak conditions, yielding an average Resilience Gap of 7.57% and highlighting variability in attack effectiveness and evaluator reliability.
arXiv AI
Sep 7
AlcaTRAz - Anchored Tree-Rule Defense Against Jailbreaks
AlcaTRAz is a prompt‑level defense that uses rule trees to insert controlled character‑level perturbations into input text, disrupting jailbreak attacks without modifying or retraining the target LLM. It operates solely on the input, making it suitable for black‑box deployments, and was evaluated on 33 open‑weight models and 22 jailbreak types, outperforming three baseline defenses in 73.4 % of model‑attack combinations. While it significantly reduces high‑severity jailbreak success, it does not eliminate it and is intended as one layer of a broader defense strategy.