The paper investigates whether the similarity score (victim‑space target alignment, VTS) reported by adversarially attacked vision‑language models actually indicates that the target text can be recovered by an independent model. Using a new measurement instrument that includes supervised judges, blend controls, shuffled negatives, and a reference level, the authors conduct two preregistered studies across six contrastive encoders and three perturbation budgets. Robustly trained encoders (FARE, TeCoA, PMG, TRADES) transfer more independent evidence than vanilla CLIP or SigLIP, yet none reach the blend‑derived reference level, leaving practical recovery uncertain; VTS is only informative within a fixed robust encoder, prompting the authors to propose a new reporting protocol.
arXiv:2610.06977v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) remain vulnerable to transferable adversarial examples, especially in black-box settings where only open-sou...
The paper demonstrates that a targeted adversarial perturbation can reduce a vision‑language model’s training loss to near zero for a fixed target caption, yet the same model, when generating freely, still produces the correct description. This phenomenon, termed the train/inference gap, is traced to a single autoregressive step where the target token’s rank is fixed across all images, and further analysis shows that the language decoder, rather than the visual encoder, determines whether the corrupted signal is amplified or suppressed. The study uses a controlled two‑stage PGD attack on Qwen2.5‑VL‑7B‑Instruct and evaluates the effect on 200 held‑out COCO images, revealing that adversarial robustness in autoregressive VLMs largely depends on the language decoder’s prior. whyItMatters":"The findings suggest that defenses and faithfulness evaluations for deployed vision‑language models should focus on the language decoder rather than the visual encoder, as the former is the key determinant of robustness to adversarial perturbations."
The paper proposes Local Appearance Purification (LAP), a method that refines the alignment target for diffusion models by removing locally predictable content and focusing on the residual. LAP improves generation quality across multiple pretrained encoders, notably reducing FID from 53.1 to 42.3 on ImageNet-256 with MAE. The authors provide a theoretical framework linking alignment effectiveness to the information emphasized by the target and introduce a constant‑readout test to predict purification benefits.
arXiv:2606. 10571v1 Announce Type: cross Abstract: Adversarial examples reveal vulnerabilities in Vision-Language Pre-training (VLP) models and provide insights for improving robustness.
arXiv:2609.38298v1 Announce Type: new Abstract: Vision Language Models (VLMs) are widely deployed in safety-critical scenarios, and understanding to which extent they can be controlled by adversarial...
The paper introduces a method for selecting the best vision‑language model for a downstream task by analyzing the internal dynamics of the visual encoder. It represents each task with layer‑wise conductance and uses an entropy‑regularized alignment to derive a target‑conditioned block importance distribution. The proposed Directional Conductance Divergence (DCD) metric captures asymmetric transferability, enabling accurate prediction of model rankings without direct inference, and achieves a 14.7% NDCG@5 improvement over SWAB on 48 VLMs across 21 datasets.
The paper introduces Shift‑Reliance Alignment (SRA), a method for ranking vision‑language models (VLMs) based on how much they rely on specific cues such as scene context or text. By editing source images to mislead these cues and measuring model responses, SRA predicts which VLMs will be misled by physical sticker attacks and which will hallucinate on unedited photographs. In pre‑registered tests, SRA selects models that reduce error on challenging datasets like Objects365 and OpenImages, outperforming selection by source error and model confidence on the CueDial benchmark.
Multi-modal Large Language Models (MLLMs) achieve strong performance on vision-language tasks, but incorporating visual inputs through a vision encoder (e. g.
arXiv:2609.26081v1 Announce Type: new Abstract: Fixed-budget robustness evaluation can select the wrong frozen vision encoder. An encoder that survives a shallow attack may lose most of that robustne...
arXiv:2605.31192v2 Announce Type: replace Abstract: Generalizable deepfake detection requires complementary forensic and semantic visual evidence. Specialist encoders capture subtle manipulation trac...
arXiv:2609.10002v1 Announce Type: new Abstract: Deepfake detectors remain vulnerable to transfer-based black-box attacks, in which adversarial examples are generated on a source surrogate model and t...
The paper introduces Inverse Knowledge Distillation (IKD), an attack‑agnostic technique that enhances adversarial transferability by maximizing the discrepancy between benign and adversarial prediction distributions on a surrogate model. IKD employs a CE/KL‑equivalent soft‑label objective to push adversarial predictions away from a fixed benign anchor, leveraging Fisher‑sensitive surrogate directions. The authors provide theoretical analysis showing CE and KL induce identical gradients, derive a lower bound on Fisher‑subspace overlap, and demonstrate through extensive ImageNet experiments that IKD consistently improves black‑box attack performance across CNN, ViT, and defended models.