arXiv Computer Vision October 9, 2026 By Illia Volkov, Nikita Kisel, Klara Janouskova, Jiri Matas

Image Recognition with Vision and Language Embeddings of VLMs

Read the original on arXiv Computer Vision →

The paper evaluates how vision-language models (VLMs) perform in image classification tasks, comparing language-guided and vision-only approaches across various dual-encoder VLMs such as SigLIP 2 and RADIOv2.5 on ImageNet-1k and its label-corrected variant. It identifies key factors influencing accuracy—including prompt design, class diversity, k‑NN neighbors, and reference set size—and demonstrates that language and vision provide complementary strengths. A simple, learning‑free fusion method based on per‑class precision is proposed to further improve classification performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision

5d ago

Multimodal Large Language Models as Image Classifiers

The paper investigates how evaluation protocols and ground‑truth quality affect the classification performance of Multimodal Large Language Models (MLLMs). It identifies and corrects key issues such as discarded out‑of‑list outputs, inflated distractor choices, and poor open‑world mapping, and shows that design choices like batch size, image ordering, and text encoder selection significantly influence accuracy. Using a multilabel reannotation of 625 ImageNet‑1k classes (ReGT), the study finds that corrected labels can boost MLLM performance by up to 10.8%, narrowing the gap with supervised models, and demonstrates that MLLMs can assist human annotators in about half of difficult cases.

By Nikita Kisel, Illia Volkov, Klara Janouskova, Jiri Matas

llms multimodal

arXiv Computer Vision

Sep 1

GATE: Reliability-Gated Gaussian Evidence Fusion for Training-Free Test-Time Adaptation of Vision-Language Models

arXiv:2608.29395v1 Announce Type: new Abstract: Vision-language models such as CLIP and SigLIP provide strong zero-shot recognition, but their predictions can degrade when deployed on target data tha...

By Pedram MohajerAnsari, Amir Salarpour, Run Wang, Mert D. Pes\'e

llms multimodal benchmarks

arXiv Computer Vision

Oct 1

Front-to-Back: Benchmarking Vision-Language Models for Asymmetric Cross-View Vehicle Re-Identification

arXiv:2609.39492v1 Announce Type: new Abstract: Matching the same vehicle across front and rear cameras is difficult because the cameras do not share a view and the vehicle's appearance changes subst...

By Moseli Mots'oehli, Thulani Babeli

llms multimodal benchmarks

arXiv Computer Vision

Oct 1

Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs

arXiv:2609.38362v1 Announce Type: new Abstract: Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining dis...

By Hung-Jen Chen, Yu-Heng Ho, Ting-Yao Huang, Po-Hsiang Hsu, Li-Yu Chen, Chun-Yi Lee, Min Sun

llms multimodal benchmarks

arXiv Computer Vision

Sep 30

VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training

arXiv:2609.36648v1 Announce Type: new Abstract: Vision-language pre-training has reshaped image clustering, giving rise to language-assisted image clustering (LaIC), which leverages textual semantics...

By Yuanwei Hu, Bo Peng, Yuheng Jia, Xinting Hu, Yadan Luo, Wenjie Zhu

llms multimodal benchmarks safety

arXiv AI

Aug 20

From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model

arXiv:2608. 18339v1 Announce Type: cross Abstract: Vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities yet remain sensitive to real-world distribution shifts during inference.

By Qi Yu, Zhichen Zeng, Katherine Tieu, Xiyuan Yang, Ruizhong Qiu, Yuchen Yan, Lihui Liu, Yanjun Zhao, Lingjie Chen, Jingrui He, Hanghang Tong

llms rag computer-vision multimodal benchmarks safety

One email a morning, machine-written

Related stories

Multimodal Large Language Models as Image Classifiers

GATE: Reliability-Gated Gaussian Evidence Fusion for Training-Free Test-Time Adaptation of Vision-Language Models

Front-to-Back: Benchmarking Vision-Language Models for Asymmetric Cross-View Vehicle Re-Identification

Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs

VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training

From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model