arXiv Computer Vision

Beyond Static Cues: Unified Dynamic Detection of MLLM Backdoors via Dual-Flow Trajectory Analysis

arXiv Computer Vision
3d ago

FusionAudit: Pathway-Conditioned Robustness Auditing for Native Multimodal Models

FusionAudit introduces PaRMA, a pathway-conditioned robustness auditing framework for native multimodal vision‑language models. It evaluates two image‑level interventions—pixel perturbation and rendered‑text insertion—to expose architecture‑specific blind spots. The study applies diagnostic‑motivated attacks (MANTIS and Nat‑Evid) to 18 open‑weight VLMs, revealing that legacy single‑attack rankings are unstable and that PaRMA uncovers higher residual risk than standard benchmarks.

By Jiaming Zhang, Zherui Li, Siqi Guo, Junhao Dong, Kun Wang, Wei Yang Bryan Lim
arXiv Machine Learning
Aug 14

Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Models

arXiv:2604. 05809v2 Announce Type: replace-cross Abstract: This paper presents Text-Guided Backdoor (TGB), an adjustable backdoor attack against multimodal pretrained models that uses natural-word triggers, namely words that can naturally occur in ordinary textual inputs.

By Yiyang Zhang, Chaojian Yu, Ziming Hong, Yuanjie Shao, Qinmu Peng, Tongliang Liu, Xinge You
arXiv AI
Oct 1

Learning Normal Diffusion Dynamics for Backdoor Defense in Text-to-Image Models

The paper introduces Normal Diffusion Dynamics Learning (NDDL), a defense framework for text-to-image diffusion models that learns normal transition dynamics from benign samples. By modeling structured, timestep‑dependent patterns across cross‑attention, latent, and noise spaces, NDDL detects backdoor attacks through deviations between observed and predicted diffusion trajectories. It also localizes triggers without prior knowledge by substituting low‑semantic words, and experiments show its effectiveness across diverse attacks.

By Junjian Li, Xiaolong Liu, Peng Sun, Liantao Wu, Linghan Chen, Yudong Gao, Honglong Chen
arXiv AI
Aug 20

Breaking the weakest link to evade vision language models

The paper investigates how Vision Language Models (VLMs) can be fooled by small, human‑imperceptible changes to images. It introduces a gradient‑based attack that targets only the vision encoder, reducing computational cost while still effectively disrupting both untargeted and targeted multimodal alignment. Experiments on open‑source VLMs such as Qwen2.5‑VL, Granite‑Vision, FastVLM, and Phi‑3.5‑Vision demonstrate that these perturbations can significantly alter the models’ textual outputs.

By Ilan Zini, Boussad Addad, Katarzyna Kapusta
arXiv AI
Oct 9

Token by Token, Compromised: Backdoor Vulnerabilities in Unified Autoregressive Models

Unified autoregressive models (UAMs) generate both text and image tokens in a single pass, simplifying multimodal generation but introducing new vulnerabilities. The authors present the Token by Token Backdoor Attack (ToBAC), the first backdoor attack targeting UAMs, which can be executed via data-based or model-based poisoning. ToBAC can manipulate both visual outputs and accompanying text, enabling subtle triggers such as the word "cool" to induce brand promotion or ideological influence in a majority of generations, with success rates up to 63.1% against JanusPro.

By Tobias Braun, Jonas Henry Grebe, Hossein Shakibania, Anna Rohrbach, Marcus Rohrbach
arXiv AI
Jun 8

Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMs

arXiv:2601. 12359v1 Announce Type: cross Abstract: Prompt injection attacks have become an increasing vulnerability for LLM applications, where adversarial prompts exploit indirect input channels such as emails or user-generated content to circumvent alignment safeguards and induce harmful or unintended outputs.

By Anirudh Sekar, Mrinal Agarwal, Rachel Sharma, Akitsugu Tanaka, Jasmine Zhang, Arjun Damerla, Kevin Zhu
arXiv AI
Oct 2

UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models

UniGuardian is a training‑free detector for large language models that jointly identifies prompt injection, backdoor, and adversarial attacks—collectively called Prompt Trigger Attacks (PTA). It measures how structured prompt perturbations shift the model’s output distribution and uses a single‑forward strategy to detect attacks while generating text in a shared batched forward pass. Experiments show that UniGuardian accurately and efficiently identifies trigger‑activated prompts in LLMs.

By Huawei Lin, Yingjie Lao, Tony Geng, Tan Yu, Weijie Zhao