arXiv AI August 26, 2026 By Harry Owiredu-Ashley

ADVERSA: Measuring Multi-Turn Guardrail Degradation and Judge Reliability in Large Language Models

The paper introduces ADVERSA, an automated red‑teaming framework that evaluates large language model safety over multiple turns by tracking continuous compliance trajectories instead of binary jailbreak outcomes. Using a fine‑tuned 70B attacker model and a structured 5‑point rubric, the authors conduct controlled experiments on three frontier victim models, measuring guardrail degradation and judge reliability through a triple‑judge consensus. Results show a 26.7% jailbreak rate with most breaches occurring early, and the study documents inter‑judge agreement, attacker drift, and attacker refusals as key factors affecting safety assessment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 24

AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability

arXiv:2606. 24589v1 Announce Type: new Abstract: Scaling adversarial evaluation of large language models requires both a method for generating hard inputs and a reliable way to confirm that resulting failures are real.

By Khanak Khandelwal (Indian Institute of Technology Jodhpur)

llms agents safety

arXiv Machine Learning

Aug 4

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.

By Ren-Jian Wang, Ke Xue, Zeyu Qin, Ziniu Li, Sheng Tang, Hao-Tian Li, Shengcai Liu, Zhi Yu, Yuanpeng Tan, Chao Qian

llms rag safety

arXiv AI

Jun 17

A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models

arXiv:2606. 18193v1 Announce Type: cross Abstract: We evaluate the adversarial robustness of two frontier large language models (LLMs) developed by Anthropic, Fable 5 and Opus 4.

By Nicola Franco

llms safety

arXiv AI

Jul 20

Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs

arXiv:2508. 10029v3 Announce Type: replace-cross Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations.

By Wenpeng Xing, Bohan Yang, Mohan Li, Chunqiang Hu, Haitao Xu, Ningyu Zhang, Bo Lin, Meng Han

llms multimodal benchmarks safety

arXiv AI

Aug 18

TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation

arXiv:2608. 15594v1 Announce Type: new Abstract: Multi-turn jailbreak attacks have emerged as a critical safety threat to LLMs, as harmful objectives are decomposed across a sequence of apparently benign turns to bypass guardrails.

By Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu, Ruihong Huang

llms robotics fine-tuning multimodal benchmarks safety

arXiv AI

Jun 9

Self-Mined Hardness for Safety Fine-Tuning

arXiv:2605. 03226v2 Announce Type: replace-cross Abstract: Safety fine-tuning of language models typically requires a curated adversarial dataset.

By Prakhar Gupta, Garv Shah, Donghua Zhang

llms fine-tuning multimodal safety

One email a morning, machine-written

Related stories

AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models

Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs

TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation

Self-Mined Hardness for Safety Fine-Tuning