arXiv AI

Which Image Property Carries the Jailbreak? A Controlled Dissection of Image-to-Text Jailbreaks

The study investigates which image properties enable jailbreak attacks on multimodal models by analyzing four attack families across 313 prompts and five models, including an appendix evaluation of InternVL3.5-8B. It finds that attack images significantly increase success rates compared to benign images, while factors such as tile entropy, JPEG size, and tile count do not reliably predict attack tiles. Only the E4 manipulation, which removes query-specific relatedness, shows a measurable effect on attack success rates, though overall conclusions depend on the rubric judge used.

arXiv AI
Aug 11

Decoy Images Amplify Caption-Mediated Defenses Against Encoded Jailbreaks

arXiv:2608. 01043v2 Announce Type: replace-cross Abstract: We report a counter-intuitive interaction between image inputs and existing black-box defenses on Vision--Language Models (VLMs): pairing an encoded jailbreak prompt with an unrelated decoy image can sharply lower attack success rate (ASR).

By Haoyu Zhang, Xiangchen Guan, Shibo Zheng, Mohammad Zandsalimy, Shanu Sushmita
arXiv Computer Vision
Aug 31

Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Models

The paper introduces Meta-Adaptive Multimodal Jailbreaking (MAMJ), a method that jointly optimizes an attack strategy prompt and attacker weights to generate more effective jailbreaks against vision‑language models. Using an LLM‑based critique to refine the strategy and group‑level success‑rate rewards to update the weights, MAMJ achieves high attack success rates on MM‑SafetyBench, outperforming existing baselines by up to 24.1 percentage points. The learned attacker also transfers to unseen models and remains robust against typical defenses, highlighting a systemic vulnerability in current VLMs.

By Benlei Cui, Shen Pang, Yuke Wang, Xuemei Dong, Yuwen Zhai, Jingqun Tang, Haiyang Yu, Hui Xue, Longtao Huang, Haiwen Hong
arXiv AI
Sep 28

Depth, Not Breadth: Best-of-N Jailbreaking Beyond Surface Noise

The paper investigates how allocating a query budget to structural depth rather than surface variation improves jailbreak success against the SAGE self‑check defense. By using a best‑of‑N approach over a code‑completion encoding, the authors achieve 67%, 22%, and 15% success rates on three open‑weight targets—far exceeding the 4.7% and 3.0% rates of single‑draw encoding and character‑search methods. The study demonstrates that depth of encoding and breadth of variation independently undermine transform and gate defenses, and that repeated sampling can inflate perceived robustness.

By Haoyu Zhang, Hanwen Liu, Yang Chen, Shibo Zheng, Xiangchen Guan, Zhuoxi Wang, Zijian Xiao, Xiao Luo, Yi Feng, Haowen Xu, Mohammad Zandsalimy, Shanu Sushmita
arXiv AI
Aug 11

Attack Ensembles Expose a Safety-Utility Trade-off in Black-Box Guard Defenses Against Encoded VLM Jailbreaks

arXiv:2607. 26574v2 Announce Type: replace-cross Abstract: Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet a guard judges an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a classical language, code, or text rendered inside an image slips past a guard that would block it in plain language - the decode gap.

By Haoyu Zhang, Zhuoxi Wang, Shibo Zheng, Yi Feng, Xiao Luo, Zijian Xiao, Haowen Xu, Xiangchen Guan, Mohammad Zandsalimy, Shanu Sushmita
arXiv AI
Sep 28

The Plot Twist: Jailbreaking Unified Multimodal Models with a Three-Act NarrativeAttack

The paper introduces NarrativeAttack, a jailbreak framework that exploits unified multimodal models (UMMs) by embedding a malicious query within a self‑contained three‑act visual narrative. The attack uses the model’s own image generator to create setup and resolution images, hiding the malicious event as a hidden climax, and concludes with an image‑based guessing game that forces the model to select the relevant answer. Experiments demonstrate that NarrativeAttack outperforms previous methods, achieving up to 88.25% attack success rate on Gemini‑2.5‑Flash, revealing a significant safety vulnerability in UMMs.

By Shaoxiong Guo, Tianyi Du, Lijun Li, Yuyao Wu, Jie Li, Jing Shao
arXiv AI
Oct 31

Jailbreak Connectivity: Towards Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreak

The paper introduces Jailbreak Connectivity (JC), a framework that exploits low‑loss pathways in image space to generate continuous adversarial paths for multimodal large language models (MLLMs). JC produces diverse jailbreak images, uses lightweight surrogate classifiers for cross‑model transfer, and applies a distribution‑level objective to create universal attacks across harmful queries. On the MM‑SafetyBench benchmark, JC achieves high attack success rates—84.01%, 85.77%, and 97.95% on Qwen3‑VL‑8B‑Instruct, InternVL3.5‑8B, and Gemma‑3‑12B‑IT respectively—outperforming existing baselines by up to 68.36 percentage points, and improves black‑box attack success on GPT‑5.6 and Gemini 3.1 Pro by 24.69 and 27.34 percentage points.

By Xu Zhang, Ziqing Hu, Shuo Han, Ren Wang
arXiv AI
Sep 2

Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate

The paper introduces the Detection Surface, a geometric framework that maps the decision boundaries of heterogeneous safety filters in text‑to‑image models. Using this insight, the authors propose CRACK, a multi‑agent debate system that iteratively mutates prompts, diagnoses layer‑specific constraints, and refines attacks to bypass composite defenses. Experiments demonstrate that CRACK can achieve attack success rates up to 99.63% while using fewer queries and preserving semantic fidelity.

By Kaiyan Wen, Shijie Zhang, Lu Yu, Guangdong Bai
arXiv AI
Aug 19

Every Picture Tells a Dangerous Story: Memory-Augmented Multi-Agent Jailbreak Attacks on VLMs

The paper introduces MemJack, a memory‑augmented multi‑agent framework that automatically generates jailbreak attacks on Vision‑Language Models (VLMs) using benign natural images as visual anchors. MemJack discovers visual anchors, camouflages them semantically, evaluates responses, repairs via reflection, and replans dynamically, forming a closed‑loop attack pipeline. The authors also create MemJack‑Bench, a dataset of over 113,000 interactive multimodal jailbreak trajectories, and show that MemJack achieves a 71.48% attack success rate against Qwen3‑VL‑Plus, reaching 90% under extended budgets, outperforming other baselines on natural‑image evaluation.

By Jianhao Chen, Haoyang Chen, Hanjie Zhao, Haozhe Liang, Zheng Wang, Tieyun Qian