The paper investigates privacy risks in Vision Transformer (ViT) split‑inference systems that use token reduction and token shuffling to lower computation and communication costs. It shows that even after token shuffling, transmitted token embeddings still contain enough positional information for a new attack, the Spatially Aligned Reconstruction Attack (SARA), which predicts token positions, restores spatial layout, fills missing embeddings with a masked autoencoder, and reconstructs the input image. While token reduction offers stronger protection, significant leakage remains when retained tokens preserve semantic and positional cues, and the authors propose a lightweight edge‑side defense that removes positional embeddings and adapts transformer blocks via knowledge distillation to reduce SARA’s effectiveness without harming downstream accuracy.
By Stefano Leggio, Giulio Rossolini, Alessandro Biondi
arXiv:2607. 00174v1 Announce Type: cross Abstract: We present a black-box model-stealing attack that recovers private vision-tokenizer configurations of deployed vision-language models (VLMs), including the visual patch size and input preprocessing pipeline.
By Kai Hu, Akash Bharadwaj, Weichen Yu, Matt Fredrikson
Vision Transformers (ViTs) are increasingly used in split-inference systems, where edge devices transmit intermediate token representations to a remote cloud. In this setting, token reduction lowers c...
arXiv:2512. 21815v4 Announce Type: replace-cross Abstract: Vision-language models (VLMs) achieve remarkable performance but remain vulnerable to adversarial attacks.
By Mengqi He, Xinyu Tian, Xin Shen, Jinhong Ni, Shu Zou, Zhaoyuan Yang, Jing Zhang
arXiv:2607. 25479v1 Announce Type: cross Abstract: Vision--Language Models (VLMs) are increasingly deployed through a model supply chain in which pretrained checkpoints, architecture definitions, text encoders, and exported computation graphs are distributed by third parties and reused across downstream services.
By Maria Rosaria Briglia, Igor Maljkovic, Antonio Emanuele Cin\`a, Luca Oneto, Iacopo Masi, Fabio Roli
arXiv:2602.08136v2 Announce Type: replace-cross
Abstract: Vision-Language Models (VLMs) are now a core part of modern AI. Recent work proposed several visual jailbreak attacks using single/ holistic...
By Md Rafi Ur Rashid, MD Sadik Hossain Shanto, Vishnu Asutosh Dasu, Shagufta Mehnaz
arXiv:2609.39134v1 Announce Type: new
Abstract: Visual-token compression improves the efficiency of large vision-language models, but can expose failures that full-token evaluation misses. We study a...
By Shilinlu Yan, Bowen Chen, Yuechen Zhang, Zhenhong Zhou, Li Sun, Sen Su
The paper exposes a hidden vulnerability in Vision Mixture-of-Experts (MoE) models that use capacity-bounded token dispatch, which varies with batch size. It presents a three-phase backdoor attack: injecting a backdoor into an early MoE layer, training a neutralizer in a deeper layer to suppress it under normal capacity, and then adjusting the batch-adaptive capacity factor so that the neutralizer is disabled when large batches are used at deployment. Experiments on V-MoE and Swin-MoE show high attack success rates (76‑87%) for large batches while keeping the attack dormant and undetected during small-batch audits, evading several state‑of‑the‑art defenses.
By Xiaocheng Zou, Tiancheng Zheng, Xiaolin Xu, Ruyi Ding
Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets. This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase computation without necessarily degrading accuracy.
The paper investigates how Vision Language Models (VLMs) can be fooled by tiny, human‑imperceptible changes to images. It introduces a gradient‑based attack that targets only the vision encoder, reducing computational cost while still effectively disrupting both untargeted and targeted multimodal interpretations. Experiments on open‑source VLMs such as Qwen2.5‑VL, Granite‑Vision, FastVLM, and Phi‑3.5‑Vision demonstrate that these small perturbations can dramatically alter the models’ textual outputs.
The paper investigates how Vision Language Models (VLMs) can be fooled by small, human‑imperceptible changes to images. It introduces a gradient‑based attack that targets only the vision encoder, reducing computational cost while still effectively disrupting both untargeted and targeted multimodal alignment. Experiments on open‑source VLMs such as Qwen2.5‑VL, Granite‑Vision, FastVLM, and Phi‑3.5‑Vision demonstrate that these perturbations can significantly alter the models’ textual outputs.
By Ilan Zini, Boussad Addad, Katarzyna Kapusta
arXiv:2607. 22716v1 Announce Type: cross Abstract: In this paper, we show for the first time that visual token pruning enhances the robustness of Multimodal Large Language Models (MLLMs), mitigating vulnerabilities such as jailbreak attacks and hallucinations.
By Shishen Gu, Jiequan Cui, Wenbo Hu, Zenglin Shi, Zhenzhen Hu, Richang Hong