← Back to all news
arXiv AI By Roman Belaire, Arunesh Sinha, Pradeep Varakantham

There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement Learning

Read the original on arXiv AI →

The paper introduces a new red‑teaming framework that treats adversarial dialogue with large language models as a Markov Decision Process, employing hierarchical reinforcement learning to handle sparse rewards and long‑horizon planning. By using a token‑level harm reward, the generative agent learns diverse, multi‑turn attacks that consistently uncover vulnerabilities beyond existing baselines. This approach achieves state‑of‑the‑art results and reframes LLM red‑teaming as a principled, trajectory‑based process.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 1

Learning diverse attacks on large language models for robust red-teaming and safety tuning

arXiv:2405.18540v3 Announce Type: replace-cross Abstract: Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of larg...

By Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Esmeralda S. Whitammer, Moksh Jain
llmsreinforcement-learningfine-tuningsafety
More like this →
arXiv AI
Sep 1

T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search

The paper introduces T-MAP, a trajectory‑aware evolutionary search technique designed to red‑team large language model agents by exploiting vulnerabilities that arise during multi‑step tool execution. Unlike traditional methods that focus on harmful text, T‑MAP uses execution trajectories to generate adversarial prompts that bypass safety guardrails and achieve harmful objectives through actual tool interactions. Experiments across various Model Context Protocol environments show that T‑MAP outperforms baseline methods in attack realization rate and remains effective against advanced models such as GPT‑5.2, Gemini‑3‑Pro, Qwen3.5, and GLM‑5.

By Hyomin Lee, Sangwoo Park, Yumin Choi, Sohyun An, Seanie Lee, Sung Ju Hwang
llmsagentssafety
More like this →