arXiv AI July 14, 2026 By Yi Ting Shen, Kentaroh Toyoda, Alex Leung

AMT-X: Phase-Structured Multi-Turn Red-Teaming with Checklist-Gated Evaluation

arXiv:2607. 11151v1 Announce Type: cross Abstract: Safety evaluation of large language models (LLMs) relies largely on single-turn attack datasets and single-judge scoring, underestimating risk from adaptive multi-turn adversaries and reporting a single success rate that does not separate partially actionable outputs from those carrying complete operational detail.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 7

NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms

arXiv:2606. 20408v3 Announce Type: replace-cross Abstract: Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized.

By Hanwool Lee, Dasol Choi, Bokyeong Kim, Haon Park, Seung Geun Kim

llms agents benchmarks safety

arXiv AI

Jun 19

LLM agent safety, multi-turn red-teaming, jailbreak benchmarks, adversarial robustness, safety-critical systems

arXiv:2606. 20408v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized.

By Hanwool Lee, Dasol Choi, Bokyeong Kim, Seung Geun Kim, Haon Park

llms agents benchmarks safety

arXiv AI

Sep 1

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

arXiv:2603.10807v2 Announce Type: replace-cross Abstract: Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Serv...

By Fabrizio Dimino, Bhaskarjit Sarmah, Stefano Pasquali

llms multimodal benchmarks safety

arXiv Machine Learning

Sep 1

Learning diverse attacks on large language models for robust red-teaming and safety tuning

arXiv:2405.18540v3 Announce Type: replace-cross Abstract: Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of larg...

By Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Esmeralda S. Whitammer, Moksh Jain

llms reinforcement-learning fine-tuning safety

arXiv AI

Aug 12

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

arXiv:2608. 10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks.

By Zixing Chen, Xingyuan Liu, Jie Zhu, Huaixia Dou, Shuo Jiang, Junhui Li, Lifan Guo, Feng Chen, Chi Zhang

llms agents multimodal benchmarks safety

arXiv Machine Learning

Jun 9

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

arXiv:2602. 16346v4 Announce Type: replace-cross Abstract: LLM-based agents execute real-world workflows via tools and memory.

By Nivya Talokar, Ayush K Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut

llms agents benchmarks safety

One email a morning, machine-written

Related stories

NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms

LLM agent safety, multi-turn red-teaming, jailbreak benchmarks, adversarial robustness, safety-critical systems

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

Learning diverse attacks on large language models for robust red-teaming and safety tuning

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents