arXiv AI

LE-GDA: Layer Ensemble Gaussian Discriminant Analysis for Training-Free Few-Shot CLIP Adaptation

arXiv Computer Vision
Oct 16

LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation

LAMP (Layerwise Adaptive Mixing of Predictors) is a training‑free framework that fuses global and regional information from multiple encoder layers for few‑shot vision‑language adaptation. It uses a single forward pass per view to generate descriptors, then applies global and support‑calibrated regional kernel predictors that independently fit label residuals. Validation data select the optimal mixture in class‑score space, preserving native feature dimensions without learned projections, and an extended variant LAMP‑F further adapts text‑anchored class prototypes for improved accuracy across eleven datasets with ViT and ResNet backbones.

By Zhipeng Ye, Feng Jiang, Qiufeng Wang, Hao Li
arXiv Machine Learning
Oct 23

Few-shot Classification as Multi-instance Verification: Effective Backbone-agnostic Transfer across Domains

The paper proposes treating few-shot classification as a series of multiple instance verification (MIV) tasks, enabling effective transfer across domains without fine-tuning the backbone. It introduces the MIV-head, a backbone‑agnostic, computationally efficient classification head that achieves competitive accuracy on cross‑domain few‑shot image classification using frozen, pre‑trained backbones. Experiments on an extended Meta‑Dataset benchmark show that the MIV-head outperforms state‑of‑the‑art adapter methods while incurring lower adaptation cost.

By Xin Xu, Eibe Frank, Geoffrey Holmes
arXiv AI
Oct 16

UniFGVC: Universal Training-Free Few-Shot Fine-Grained Vision Classification via Attribute-Aware Multimodal Retrieval

UniFGVC introduces a training‑free framework for few‑shot fine‑grained visual classification that reframes the task as a multimodal retrieval problem. It uses a Category‑Discriminative Visual Captioner to generate attribute‑aware textual descriptions, then matches image‑description pairs with category templates via pre‑trained encoders. Experiments on 12 datasets show that UniFGVC consistently outperforms existing CLIP‑based and fully‑supervised MLLM methods, demonstrating the effectiveness of the retrieval paradigm.

By Haiyun Guo, Hongyu Guo, Xiangzhao Hao, Jiarui Guo, Jinqiao Wang, Tat-Seng Chua
arXiv AI
Sep 25

Domain Recentering and Confidence-Weighted Prior Calibration for Vision-Language Models

The paper introduces Domain Recentering with Confidence Calibration (DRC), a training‑free technique that adapts CLIP to unlabeled target images by fitting a Gaussian mixture and subtracting a posterior‑weighted average of component means from each embedding. It further corrects residual class bias using a log‑prior adjustment based on confidence‑weighted predictions. DRC outperforms other methods, raising average accuracy on cross‑domain datasets by 4.13 and 5.07 points over zero‑shot CLIP for ViT‑B/16 and ResNet‑50, and maintains gains under ImageNet distribution shifts.

By Youngeun Seol, Jimin Shin, Heeseo Yoon, Uiwon Hwang
Hugging Face Trending Papers
Sep 24

Domain Recentering and Confidence-Weighted Prior Calibration for Vision-Language Models

The paper introduces Domain Recentering with Confidence Calibration (DRC), a training‑free technique that adapts CLIP to unlabeled target images by fitting a Gaussian mixture and applying posterior‑weighted mean subtraction, followed by a log‑prior correction based on confidence‑weighted predictions. DRC improves cross‑domain accuracy, surpassing zero‑shot CLIP by 4.13 and 5.07 points on ViT‑B/16 and ResNet‑50, and maintains gains under ImageNet distribution shifts.

Hugging Face Trending Papers
Aug 27

G2D: Generative-to-Discriminative Collaborative Inference for Zero-Shot Image Classification

G2D is a training‑free framework that combines a discriminative model (CLIP) for broad candidate retrieval with a generative vision‑language model for fine‑grained, image‑grounded verification. By using CLIP’s top‑K shortlist and a structured prior from candidate names and probabilities, G2D focuses generative reasoning on uncertain samples, employing fixed confidence routing, entropy‑adaptive candidate sizing, and trie‑constrained decoding to produce a single valid output. Across eight benchmarks, G2D achieves an average accuracy of 68.85%, outperforming both CLIP (59.35%) and the standalone generative model (63.11%), and it also transfers effectively to other models such as DCLIP, WaffleCLIP, and CuPL.

arXiv Computer Vision
Aug 28

G2D: Generative-to-Discriminative Collaborative Inference for Zero-Shot Image Classification

The paper introduces G2D, a training‑free framework that combines a discriminative model (CLIP) for broad candidate retrieval with a generative vision‑language model for fine‑grained, image‑grounded verification. By using CLIP’s top‑K shortlist and a structured prior from candidate names and probabilities, G2D focuses generative reasoning on uncertain samples, achieving an average accuracy of 68.85% across eight benchmarks—higher than both CLIP alone (59.35%) and the standalone generative model (63.11%). The approach also adapts to various generator configurations and extends to other models such as DCLIP, WaffleCLIP, and CuPL.

By Zehua Hao, Fang Liu, Qinliang Wang, Yaoyang Du, Xinyan Huang, Puhua Chen