arXiv Machine Learning
Learning diverse attacks on large language models for robust red-teaming and safety tuning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 4
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.
arXiv AI
Aug 12
Generating Attacks for LLMs with GFlowNets
arXiv:2608. 10171v1 Announce Type: new Abstract: The rapid advancement of Large Language Models (LLMs) has facilitated their ubiquitous integration into various domains, leading to widespread adoption.
arXiv AI
Sep 1
T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search
The paper introduces T-MAP, a trajectory‑aware evolutionary search technique designed to red‑team large language model agents by exploiting vulnerabilities that arise during multi‑step tool execution. Unlike traditional methods that focus on harmful text, T‑MAP uses execution trajectories to generate adversarial prompts that bypass safety guardrails and achieve harmful objectives through actual tool interactions. Experiments across various Model Context Protocol environments show that T‑MAP outperforms baseline methods in attack realization rate and remains effective against advanced models such as GPT‑5.2, Gemini‑3‑Pro, Qwen3.5, and GLM‑5.
arXiv AI
Jun 9
Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
arXiv:2606. 09701v1 Announce Type: cross Abstract: AI red teaming must continually adapt to evolving attackers and defenders.
arXiv Machine Learning
Aug 5
Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning
arXiv:2508. 20697v4 Announce Type: replace Abstract: As large language models (LLMs) continue to grow in capability, so do the risks of harmful misuse through fine-tuning.
arXiv Machine Learning
Jul 1
Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance
arXiv:2605. 00553v3 Announce Type: replace Abstract: Large Language Model (LLM) Red-Teaming, which proactively identifies vulnerabilities of LLMs, is an essential process for ensuring safety.