arXiv Computer Vision August 27, 2026 By Shuang Liang, Zhihao Xu, Jiaqi Weng, Jialing Tao, Hui Xue, Xiting Wang

Learning to Detect Unseen Jailbreak Attacks in Large Vision-Language Models

Read the original on arXiv Computer Vision →

The paper introduces Learning to Detect (LoD), a framework for identifying unseen jailbreak attacks in Large Vision‑Language Models without relying on attack data or hand‑crafted heuristics. LoD extracts layer‑wise safety representations via Multi‑modal Safety Concept Activation Vectors and compresses them into a one‑dimensional anomaly score using a Safety Pattern Auto‑Encoder. Experiments show that LoD achieves state‑of‑the‑art AUROC across diverse unseen attacks on multiple LVLMs while improving efficiency.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Sep 1

Robustness of Vision Language Models Against Split-Image Harmful Input Attacks

arXiv:2602.08136v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are now a core part of modern AI. Recent work proposed several visual jailbreak attacks using single/ holistic...

By Md Rafi Ur Rashid, MD Sadik Hossain Shanto, Vishnu Asutosh Dasu, Shagufta Mehnaz

llms reinforcement-learning fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jun 12

FENCE: A Financial and Multimodal Jailbreak Detection Dataset

arXiv:2602. 18154v2 Announce Type: replace-cross Abstract: Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs).

By Mirae Kim, Seonghun Jeong, Youngjun Kwak

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 30

Sparse Autoencoders are Capable LLM Jailbreak Mitigators

arXiv:2602. 12418v2 Announce Type: replace-cross Abstract: Jailbreak attacks remain a persistent threat to large language model safety.

By Yannick Assogba, Jacopo Cortellazzi, Javier Abad, Pau Rodriguez, Xavier Suau, Arno Blaas

llms diffusion safety

arXiv Computation and Language

Aug 25

Text-Anchored Semantic Perturbations for Transferable Jailbreak Attacks on Multimodal Large Language Models

The paper introduces TA-SPA, a black‑box jailbreak method for multimodal large language models that generates transferable perturbations in a text‑anchored semantic space. It combines Text‑Anchored Semantic Factorization (TASF) to separate cross‑modal semantic factors from modality‑specific residuals with Semantic‑Preserving Augmentation (SPA) to diversify harmful target anchors while maintaining semantic consistency. Experiments demonstrate strong attack effectiveness and transferability to commercial MLLMs, with competitive performance against representative defenses.

By Wenyun Li, Guiping Cao, Xiangyuan Lan, Zheng Zhang

llms multimodal safety

arXiv AI

Jun 3

Black-box, Adaptive, Efficient, Transferable, Harmful, Applicable... Attacks Are All You Need to Break LLMs

arXiv:2606. 03647v1 Announce Type: cross Abstract: Accurately evaluating adversarial robustness is a longstanding challenge.

By Vincent Limbach, Jonas Dornbusch, David L\"udke, Stephan G\"unnemann, Leo Schwinn

llms diffusion fine-tuning benchmarks safety

arXiv Computer Vision

Sep 23

Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application

arXiv:2503.06989v5 Announce Type: replace-cross Abstract: Recently, Multimodal Large Language Models (MLLMs) have demonstrated their superior ability in understanding multimodal content. However, the...

By Wenzhuo Xu, Zhipeng Wei, Xiongtao Sun, Zonghao Ying, Deyue Zhang, Dongdong Yang, Xiangzheng Zhang, Quanchen Zou

llms fine-tuning multimodal safety

One email a morning, machine-written

Related stories

Robustness of Vision Language Models Against Split-Image Harmful Input Attacks

FENCE: A Financial and Multimodal Jailbreak Detection Dataset

Sparse Autoencoders are Capable LLM Jailbreak Mitigators

Text-Anchored Semantic Perturbations for Transferable Jailbreak Attacks on Multimodal Large Language Models

Black-box, Adaptive, Efficient, Transferable, Harmful, Applicable... Attacks Are All You Need to Break LLMs

Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application