arXiv Machine Learning By Jiazhen Huang, Xiao Chen, Zhiming Liu, Yaru Sun, Jingyan Jiang, Zhi Wang

What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

Read the original on arXiv Machine Learning →

arXiv:2606. 14299v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) such as CLIP have become a standard backbone for open-vocabulary recognition, yet their zero-shot predictions remain vulnerable to distribution shifts encountered at deployment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Sep 10

To Adapt or Not to Adapt? Selective Adaptation for Vision-Language Models

The paper introduces selective adaptation for vision‑language models, questioning whether test‑time adaptation (TTA) should always be applied. By analyzing per‑sample predictions before and after adaptation, the authors find that many adaptations are negligible or even harmful, flipping correct predictions. They propose Cross‑Augmentation Similarity (CAS), which skips adaptation when predictions across augmented views are highly similar, achieving comparable or better accuracy while reducing adaptation by up to 85%.

By Siru Jiang, Yuwei Liang, Jian Liang, Ran He, Tieniu Tan
arXiv AI
Aug 28

Subspace Alignment for Vision-Language Model Test-time Adaptation

The paper introduces SubTTA, a test-time adaptation method for vision‑language models that aligns the semantic subspaces of visual and textual modalities to improve zero‑shot predictions. It addresses two issues: the modality gap caused by distribution shifts and visual nuisance that masks task‑specific semantics. By minimizing chordal distance between principal subspaces and projecting visual features onto a task‑specific textual subspace, SubTTA refines decision boundaries and achieves an average 2.24% improvement over existing TTA methods.

By Zhichen Zeng, Wenxuan Bao, Xiao Lin, Ruizhong Qiu, Tianxin Wei, Xuying Ning, Yuchen Yan, Chen Luo, Monica Xiao Cheng, Jingrui He, Hanghang Tong