← Back to all news
arXiv AI By Jekaterina Novikova, Yuqing Huang, Jack Ji, Sagarika Suresh, Jithin Pradeep, Lovedeep Gondara

Reject, Remember, Retry: Why Multi-Turn Attackers Should Remember Their Mistakes

Read the original on arXiv AI →

The paper introduces $R^3$ (Reject, Remember, Retry), a black-box multi-LLM framework that improves multi-turn jailbreak attacks by conditioning on a compact state instead of the full conversation transcript. In $R^3$, failed attack turns are removed from the target’s context but stored by the attacker along with the reason for failure, creating an asymmetric memory. Experiments on two benchmarks and six target LLMs show that $R^3$ outperforms state‑of‑the‑art baselines by up to 53% in ASR@5 under matched query budgets, and even a low‑capacity attacker using $R^3$’s state can surpass high‑capacity attackers without it by up to 34% in ASR@5.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 26

$M^3$: A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process

$M^3$ is an automated red‑teaming framework that models multi‑turn jailbreaking as a finite‑horizon Markov Decision Process. It lets an attacker LLM interact with a target LLM while using a state LLM to maintain a compact internal state. Across two benchmarks and six target models, $M^3$ outperforms existing methods, boosting attack success rates by up to 53% at five turns and improving consistency and risk escalation metrics.

By Yuqing Huang, Jack Ji, Jithin Pradeep, Sagarika Suresh, Jekaterina Novikova
llmsreinforcement-learningmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 19

Multiple Turns, Short Horizons: Measuring Effective-Horizon Compression in Jailbreaks

The paper investigates how much of a multi-turn jailbreak attack’s query history is necessary to achieve the same progress. By removing early attacker queries and re-running the remaining sequence, the authors define the effective horizon as the shortest query set that reproduces the attack outcome. They find that, on average, about half of the original queries suffice, a phenomenon they term effective-horizon compression, and demonstrate that this shorter history can improve attack continuation, training efficiency, and maintain defense calibration.

By Zhipeng Wei, Michael Lee, N. Benjamin Erichson
llms
More like this →
arXiv Computation and Language
Sep 16

Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion

The paper introduces the SAST-IR framework to evaluate large language models’ robustness against persuasion attacks in a memory‑less setting, revealing a flaw called "Refusal Inertia" that masks true vulnerability. Using the CP‑Agent and a custom CounterFact‑Strict dataset, the authors demonstrate that simple, diverse attack strategies achieve a 96% success rate, while complex attacks often trigger defensive compliance. The study highlights severe brittleness in current state‑of‑the‑art models when deprived of conversation history.

By Zhuoang Cai
llmsagentsbenchmarkssafety
More like this →