← Back to all news
arXiv AI By Feitong Qiao, Liren Peng, Shiming Ren, Aishwarya Jadhav, Arghavan Bahadorinejad, Marinette Chen, Muhan Zhang, Abdulaziz Suria, Gennevi Lu, Anish Das Sarma

EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities

Read the original on arXiv AI →

EvoFlint is an evolutionary atlas that maps multi‑turn vulnerabilities in large language models by treating attack discovery as a search problem rather than a generation task. It uses evolutionary quality‑diversity search to evolve phased conversation plans, employing Pareto fitness for success rate and severity, novelty search for diversity, and a generation‑level memory to incorporate model insights. The resulting risk‑indexed archive, tested on HarmBench, shows high attack success rates across models such as Claude Sonnet, GPT‑5, and Qwen3, revealing which harm categories each model’s safety training covers or misses.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Aug 4

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.

By Ren-Jian Wang, Ke Xue, Zeyu Qin, Ziniu Li, Sheng Tang, Hao-Tian Li, Shengcai Liu, Zhi Yu, Yuanpeng Tan, Chao Qian
llmsragsafety
More like this →
arXiv AI
Sep 1

T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search

The paper introduces T-MAP, a trajectory‑aware evolutionary search technique designed to red‑team large language model agents by exploiting vulnerabilities that arise during multi‑step tool execution. Unlike traditional methods that focus on harmful text, T‑MAP uses execution trajectories to generate adversarial prompts that bypass safety guardrails and achieve harmful objectives through actual tool interactions. Experiments across various Model Context Protocol environments show that T‑MAP outperforms baseline methods in attack realization rate and remains effective against advanced models such as GPT‑5.2, Gemini‑3‑Pro, Qwen3.5, and GLM‑5.

By Hyomin Lee, Sangwoo Park, Yumin Choi, Sohyun An, Seanie Lee, Sung Ju Hwang
llmsagentssafety
More like this →
arXiv AI
Oct 26

$M^3$: A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process

$M^3$ is an automated red‑teaming framework that models multi‑turn jailbreaking as a finite‑horizon Markov Decision Process. It lets an attacker LLM interact with a target LLM while using a state LLM to maintain a compact internal state. Across two benchmarks and six target models, $M^3$ outperforms existing methods, boosting attack success rates by up to 53% at five turns and improving consistency and risk escalation metrics.

By Yuqing Huang, Jack Ji, Jithin Pradeep, Sagarika Suresh, Jekaterina Novikova
llmsreinforcement-learningmultimodalbenchmarkssafety
More like this →
arXiv Machine Learning
Sep 1

Learning diverse attacks on large language models for robust red-teaming and safety tuning

arXiv:2405.18540v3 Announce Type: replace-cross Abstract: Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of larg...

By Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Esmeralda S. Whitammer, Moksh Jain
llmsreinforcement-learningfine-tuningsafety
More like this →