arXiv AI
A Cross-Architecture Audit of Direction-Based Inference-Time Defences in Vision-Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 11
Attack Ensembles Expose a Safety-Utility Trade-off in Black-Box Guard Defenses Against Encoded VLM Jailbreaks
arXiv:2607. 26574v2 Announce Type: replace-cross Abstract: Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet a guard judges an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a classical language, code, or text rendered inside an image slips past a guard that would block it in plain language - the decode gap.
arXiv Machine Learning
Jul 30
Recover, Decode, Reguard: Guard-Agnostic Defense Amplification againstEncoded VLM Jailbreaks
arXiv:2607. 26574v1 Announce Type: cross Abstract: Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet they judge an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a rare language, code, or an image of text slips past a guard that would block it in plain language -- the decode gap.
arXiv AI
Aug 11
Decoy Images Amplify Caption-Mediated Defenses Against Encoded Jailbreaks
arXiv:2608. 01043v2 Announce Type: replace-cross Abstract: We report a counter-intuitive interaction between image inputs and existing black-box defenses on Vision--Language Models (VLMs): pairing an encoded jailbreak prompt with an unrelated decoy image can sharply lower attack success rate (ASR).
arXiv Computer Vision
4d ago
Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Models
The paper introduces Meta-Adaptive Multimodal Jailbreaking (MAMJ), a method that jointly optimizes an attack strategy prompt and attacker weights to generate more effective jailbreaks against vision‑language models. Using an LLM‑based critique to refine the strategy and group‑level success‑rate rewards to update the weights, MAMJ achieves high attack success rates on MM‑SafetyBench, outperforming existing baselines by up to 24.1 percentage points. The learned attacker also transfers to unseen models and remains robust against typical defenses, highlighting a systemic vulnerability in current VLMs.
arXiv AI
Jul 24
When Are Reasoning-Based Guardrails Not Efficient? ResponseGuard: A Fast Vision-Language Guard for Real-Time Moderation
arXiv:2607. 21401v1 Announce Type: cross Abstract: A vision-language AI assistant returns its answer as a stream of generated tokens.
arXiv AI
Aug 11
SALLIE: Generation-Free Hidden-State Detection of Jailbreaks and Prompt Injections Across Text and Vision
arXiv:2604. 06247v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) and Vision-Language Models (VLMs) are vulnerable to jailbreaks and prompt injections delivered through text or images.