Prompt learning adapts vision-language models, such as CLIP, by adjusting a small set of context tokens. However, under few-shot supervision, even moderate label noise can disrupt prompt optimization....
TecoPrompt introduces a closed‑loop robust prompt‑learning framework that mitigates label noise in vision‑language models. It uses optimal transport in the CLIP semantic space to generate globally consistent pseudo‑labels, then verifies their reliability by checking trajectory stability over a K‑epoch window and an EMA‑based confidence gate. The verified labels are incorporated into prompt training via a tri‑group objective, yielding significant accuracy gains across multiple noisy datasets, such as a 0.843 accuracy on OxfordPets with 50% asymmetric noise.
By Zeyi Shao, Haowen Hua, Jiaxin Zhang, John See, Zeyd Boukhers, Cong Yang
arXiv:2607. 00684v1 Announce Type: new Abstract: The classification accuracy of pretrained Vision-Language Models (VLMs) relies on the quality of the text prompts.
By Seokhee Jin, Changhwan Sung, Sunung Mun, Hoyoung Kim, Jungseul Ok
ProCAP introduces a probabilistic cross-attentive prompt learning framework for vision-language models like CLIP, enabling improved cross-modal interaction without updating the backbone. It jointly learns visual and textual prompt tokens, linking them via stacked bidirectional multi-head cross-attention to refine each branch across prompt depth. The method incorporates Gaussian parameterization of prompt tokens, lightweight KL and L2 regularization, and a compact symmetric InfoNCE head to align image features with class-level text representations, achieving strong few-shot base-to-novel performance and competitive transfer results across multiple datasets and benchmarks.
By Hiwa Azeez Abbas, Fatemeh Daneshfar, Moloud Abdar
The paper introduces Language-driven Dense Semantic Adaptor (LDSA) for multi-label image classification with incomplete annotations. LDSA leverages multimodal pretrained CLIP models to extract prior-adaptive relationships, employing a densely contrastive adaptor for visual contrastive constraints and a language-driven interactive decoder with class-specific prompt tuning. Experiments show LDSA achieves state‑of‑the‑art performance on public benchmarks and reveals implicit semantic relationships through its learning scheme.
By Cheng Chen, Yifan Zhao, Jia Li
arXiv:2506. 10292v2 Announce Type: replace-cross Abstract: Training deep learning networks with minimal supervision has gained significant research attention due to its potential to reduce reliance on extensive labelled data.
By Ali Almutairi, Abdullah Alsuhaibani, Shoaib Jameel, Aditya Joshi, Gelareh Mohammadi, Imran Razzak