arXiv AI October 31, 2026 By Xiaogeng Liu, Xinyan Wang, Chaowei Xiao

FlipJev: Fast Black-Box Red Teaming Finds Universal Adversarial Suffixes for Typed Decision Models

The paper introduces FlipJev, a fast black-box red-teaming method that discovers universal adversarial suffixes for typed decision models—models that output a single decision rather than free text. Using a quick query-based attack without an attacker language model, the authors tested 12 datasets and found that while Jev resists common prompt injections, their suffixes can flip confident answers within minutes, affecting 43% of unseen documents. Moreover, a single suffix generated from one task can flip 90–100% of samples in other tasks, highlighting a significant vulnerability across diverse applications.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 12

Generating Attacks for LLMs with GFlowNets

arXiv:2608. 10171v1 Announce Type: new Abstract: The rapid advancement of Large Language Models (LLMs) has facilitated their ubiquitous integration into various domains, leading to widespread adoption.

By Berkay Ozcam, Irem Onen, Mehmet Fatih Amasyali, Emin Islam Tatli

llms benchmarks safety

arXiv Machine Learning

Aug 4

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

arXiv:2506. 07121v2 Announce Type: replace Abstract: Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence.

By Ren-Jian Wang, Ke Xue, Zeyu Qin, Ziniu Li, Sheng Tang, Hao-Tian Li, Shengcai Liu, Zhi Yu, Yuanpeng Tan, Chao Qian

llms rag safety

arXiv Machine Learning

Sep 1

Learning diverse attacks on large language models for robust red-teaming and safety tuning

arXiv:2405.18540v3 Announce Type: replace-cross Abstract: Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of larg...

By Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Esmeralda S. Whitammer, Moksh Jain

llms reinforcement-learning fine-tuning safety

arXiv AI

Jul 22

A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

arXiv:2606. 25476v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated remarkable performance across natural language processing tasks, yet their deployment in high-stakes applications raises critical concerns regarding reliability, safety, and trustworthiness.

By Abrar Alotaibi, Raed Mughus, Moataz Ahmed

llms nlp safety

arXiv AI

Jun 16

Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment

arXiv:2606. 15441v1 Announce Type: cross Abstract: Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.

By Lipeng He, Yihan Wang, Jiawen Zhang, N. Asokan

llms rag agents reinforcement-learning multimodal benchmarks safety

Hugging Face Trending Papers

Jun 13

Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment

Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense.

llms rag agents reinforcement-learning multimodal benchmarks safety

One email a morning, machine-written

Related stories

Generating Attacks for LLMs with GFlowNets

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

Learning diverse attacks on large language models for robust red-teaming and safety tuning

A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment

Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment