arXiv:2608.29395v1 Announce Type: new
Abstract: Vision-language models such as CLIP and SigLIP provide strong zero-shot recognition, but their predictions can degrade when deployed on target data tha...
By Pedram MohajerAnsari, Amir Salarpour, Run Wang, Mert D. Pes\'e
The paper introduces UnInfo, a test‑time adaptation method for vision‑language models like CLIP that addresses image corruption—a realistic distribution shift caused by sensor conditions. UnInfo leverages uniformity‑aware confidence maximization, information‑aware loss balancing, and knowledge distillation from an EMA teacher to preserve embedding uniformity and improve zero‑shot classification accuracy. Experiments show that UnInfo outperforms existing TTA methods on corrupted image datasets.
By Kazuki Adachi, Shin'ya Yamaguchi, Tomoki Hamagami
G2D is a training‑free framework that combines a discriminative model (CLIP) for broad candidate retrieval with a generative vision‑language model for fine‑grained, image‑grounded verification. By using CLIP’s top‑K shortlist and a structured prior from candidate names and probabilities, G2D focuses generative reasoning on uncertain samples, employing fixed confidence routing, entropy‑adaptive candidate sizing, and trie‑constrained decoding to produce a single valid output. Across eight benchmarks, G2D achieves an average accuracy of 68.85%, outperforming both CLIP (59.35%) and the standalone generative model (63.11%), and it also transfers effectively to other models such as DCLIP, WaffleCLIP, and CuPL.
The paper introduces G2D, a training‑free framework that combines a discriminative model (CLIP) for broad candidate retrieval with a generative vision‑language model for fine‑grained, image‑grounded verification. By using CLIP’s top‑K shortlist and a structured prior from candidate names and probabilities, G2D focuses generative reasoning on uncertain samples, achieving an average accuracy of 68.85% across eight benchmarks—higher than both CLIP alone (59.35%) and the standalone generative model (63.11%). The approach also adapts to various generator configurations and extends to other models such as DCLIP, WaffleCLIP, and CuPL.
By Zehua Hao, Fang Liu, Qinliang Wang, Yaoyang Du, Xinyan Huang, Puhua Chen
ProCAP introduces a probabilistic cross-attentive prompt learning framework for vision-language models like CLIP, enabling improved cross-modal interaction without updating the backbone. It jointly learns visual and textual prompt tokens, linking them via stacked bidirectional multi-head cross-attention to refine each branch across prompt depth. The method incorporates Gaussian parameterization of prompt tokens, lightweight KL and L2 regularization, and a compact symmetric InfoNCE head to align image features with class-level text representations, achieving strong few-shot base-to-novel performance and competitive transfer results across multiple datasets and benchmarks.
By Hiwa Azeez Abbas, Fatemeh Daneshfar, Moloud Abdar
The paper introduces Domain Recentering with Confidence Calibration (DRC), a training‑free technique that adapts CLIP to unlabeled target images by fitting a Gaussian mixture and applying posterior‑weighted mean subtraction, followed by a log‑prior correction based on confidence‑weighted predictions. DRC improves cross‑domain accuracy, surpassing zero‑shot CLIP by 4.13 and 5.07 points on ViT‑B/16 and ResNet‑50, and maintains gains under ImageNet distribution shifts.