The paper introduces a method for selecting the best vision‑language model for a downstream task by analyzing the internal dynamics of the visual encoder. It represents each task with layer‑wise conductance and uses an entropy‑regularized alignment to derive a target‑conditioned block importance distribution. The proposed Directional Conductance Divergence (DCD) metric captures asymmetric transferability, enabling accurate prediction of model rankings without direct inference, and achieves a 14.7% NDCG@5 improvement over SWAB on 48 VLMs across 21 datasets.
arXiv:2608. 18339v1 Announce Type: cross Abstract: Vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities yet remain sensitive to real-world distribution shifts during inference.
arXiv:2608.06411v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) achieve strong performance across diverse vision-language tasks, but their efficiency is limited by...
arXiv:2608. 06411v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) achieve strong performance across diverse vision-language tasks, but their efficiency is limited by the cost of processing numerous visual tokens.
arXiv:2609.37165v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models remain brittle under visual distribution shifts, often relying on spurious correlations tied to domain-specific f...
arXiv:2606. 05758v1 Announce Type: cross Abstract: Many modern vision-language models (VLMs) build on autoregressive decoding of discrete tokens.
arXiv:2606. 15134v1 Announce Type: cross Abstract: Vision encoders for retrieval are typically trained with class-label supervision: each training pair reduces to a scalar that uniformly pushes the embedding apart or pulls it together, as if every visual attribute either differed or matched.
arXiv:2505.07675v3 Announce Type: replace-cross Abstract: Semi-supervised learning (SSL) has emerged as a practical solution for addressing data scarcity challenges by leveraging unlabeled data. Rece...
Abundant visual information strengthens vision-language model (VLM) perception, yet massive visual tokens raise inference costs. Existing visual token pruning methods rely on similarity-based guidance, which exploits pairwise text-vision and vision-vision token correlations for compression.
DiscoVL introduces a disentangled cross‑modal representation learning framework for vision‑language models, combining orthogonal adversarial regularization with a multi‑branch low‑rank residual aligner to enhance bidirectional visual‑textual interaction. The method decomposes representations into subspaces, enabling cross‑modal feedback at each layer and preventing centroid collapse through an orthogonal adversarial triplet loss. Across 15 benchmarks, DiscoVL consistently outperforms state‑of‑the‑art approaches in base‑to‑novel generalization, cross‑dataset evaluation, and few‑shot learning.
arXiv:2609.36916v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) incur high inference latency from long visual token sequences. Existing pruning methods commonly use attention...
arXiv:2609.23717v1 Announce Type: new Abstract: Global vision--language similarities compress an image and a caption into one vector, preserving semantics but not which word corresponds to which regi...
The paper investigates how the modality gap— the separation between image and text representations in contrastive vision‑language models—affects different downstream tasks. By showing that a single dominant direction accounts for most of the image‑text mean separation, the authors explain why reducing or removing this gap can improve zero‑shot classification, degrade retrieval, or restore performance depending on the task. The study provides a geometric framework that clarifies when and why gap interventions should be applied in vision‑language systems.