← Back to all news
arXiv AI By Yi Ting Shen, Kentaroh Toyoda, Alex Leung

ACEA: An Adversarial Co-Evolution Arena for Head-to-Head Red-Team and Blue-Team LLM Testing

Read the original on arXiv AI →

ACEA (Adversarial Co‑Evolution Arena) is a platform that connects pluggable red‑team and blue‑team adapters to a shared target LLM, scoring attack and defense rates with an LLM judge. It introduces a minimal HTTP protocol (ASAP) for any language, an evaluation methodology that seeds the target with canonical secrets to separate leakage from hallucination, and a real‑time game‑style visualization with detailed end‑of‑battle reports. Additionally, ACEA offers an optional in‑context improvement loop that provides advisory hints to adapters for iterative refinement across rounds.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
1d ago

AutoRedClaw: Automated Red-Teaming for Claw-Style Agents

AutoRedClaw is an automated red‑teaming framework designed for claw‑style agents such as OpenClaw, which maintain persistent configuration, memory, and reusable skills. It generates end‑to‑end attacks and the necessary environments, using a strict quality‑checking protocol and execution‑based validation to ensure realistic, effective samples. The framework produces a 330‑case benchmark, RedClaw‑Bench, achieving over 30% effective attack rate at less than $0.10 per successful sample and demonstrating higher attack success rates than existing benchmarks across multiple agent systems.

By Hao Li, Tongyan Hu, Weidi Luo, Shanghao Shi, Xiaogeng Liu, Aiden Suh, Yulong Cao, Ning Zhang, William Yeoh, Yevgeniy Vorobeychik, Chaowei Xiao
agentsmultimodalbenchmarkssafety
More like this →
arXiv AI
Aug 26

ADVERSA: Measuring Multi-Turn Guardrail Degradation and Judge Reliability in Large Language Models

The paper introduces ADVERSA, an automated red‑teaming framework that evaluates large language model safety over multiple turns by tracking continuous compliance trajectories instead of binary jailbreak outcomes. Using a fine‑tuned 70B attacker model and a structured 5‑point rubric, the authors conduct controlled experiments on three frontier victim models, measuring guardrail degradation and judge reliability through a triple‑judge consensus. Results show a 26.7% jailbreak rate with most breaches occurring early, and the study documents inter‑judge agreement, attacker drift, and attacker refusals as key factors affecting safety assessment.

By Harry Owiredu-Ashley
llmsfine-tuningsafety
More like this →
arXiv Machine Learning
Sep 10

CoRL: Co-Evolutionary Reinforcement Learning for Adaptive Indirect Prompt-Injection Attacks and Defenses

The paper introduces CoRL, a co-evolutionary reinforcement learning framework designed to defend against adaptive indirect prompt-injection attacks on tool-augmented language agents. CoRL operates in three stages—attacker fine‑tuning, bilateral Co‑PPO training, and defender fine‑tuning—using verifier‑grounded repairs to adapt to changing attack strategies. Experiments on 1,514 executions show that CoRL reduces attack success rates to 0% while improving task utility, demonstrating its effectiveness against adaptive adversaries.

By Boyang Zhang, Qingxin Xiao, Lingwei Dang, Qingyao Wu
agentsreinforcement-learningfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv Machine Learning
Aug 4

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.

By Ren-Jian Wang, Ke Xue, Zeyu Qin, Ziniu Li, Sheng Tang, Hao-Tian Li, Shengcai Liu, Zhi Yu, Yuanpeng Tan, Chao Qian
llmsragsafety
More like this →