The paper revisits the use of contrastive loss for post‑training CLIP models, showing that the reported catastrophic forgetting is mainly due to an overly large temperature parameter in the InfoNCE objective. By setting a smaller temperature, contrastive post‑training can actually improve CLIP. The authors introduce ComCLIP, a lightweight single‑epoch recipe that freezes the text encoder, trains the vision encoder with a properly tempered contrastive loss, an MSE anchoring loss, and a relational distillation loss, achieving comparable zero‑shot classification to CLIP‑Refine while significantly enhancing visual feature transferability and maintaining downstream compatibility with LLaVA.
By Zidan Wang, Yaqian Li, Xiaokai Zhang, Kaiwen Long, Kun He, Hanpeng Liu
arXiv:2610.03370v1 Announce Type: new
Abstract: CLIP's visual encoder produces only global image representations, limiting its use in region-level tasks. Existing adaptations rely on visual prompting...
By Anh-Khoa Dinh-Duc, Duc-Tai Dinh, Tam V. Nguyen, Minh-Triet Tran
arXiv:2606. 16799v1 Announce Type: cross Abstract: Existing vision-language model (VLM)-based AI-generated image quality assessment (AIGIQA) methods suffer from a fundamental semantic-distortion dimensional conflict: monolithic representations optimized for semantic discrimination inherently entangle compositional understanding with low-level perceptual sensitivity, rendering them blind to fine-grained quality degradations.
By Zijie Meng
LensVLM is an inference framework and post‑training recipe that lets Vision‑Language Models (VLMs) process compressed images of text by selectively expanding only the relevant parts back to full resolution. Using Qwen3.5‑9B‑Base, LensVLM achieves accuracy comparable to full‑text models at 4.3× compression and outperforms other compression baselines up to 10.1× across seven text QA benchmarks, while also improving performance on multimodal document and code tasks as compression increases.
By Roy Xie, Dan Friedman, Donghan Yu, Bowen Pan, Christopher Fifty, Jang-Hyun Kim, Xianzhi Du, Zhe Gan, Vivek Rathod, Bhuwan Dhingra
arXiv:2609.24564v1 Announce Type: new
Abstract: CLIP, a foundational vision-language model, has emerged as a powerful tool for open-vocabulary semantic segmentation. While freezing CLIP's text encode...
By Zelin Peng, Zhengqin Xu, Changsong Wen, Yu Huang, Yaoming Wang, Xiaokang Yang, Wei Shen
arXiv:2603.02767v4 Announce Type: replace-cross
Abstract: Image--text contrastive pretraining has become a dominant paradigm for visual representation learning, yet existing methods often yield repre...
By Hanpeng Liu, Zidan Wang, Shuoxi Zhang, Zonglin Zhao, Zihao Bo, Rinyoichi Takezoe, Kaiwen Long, Yaqian Li, Kun He
The paper investigates using raw, unlabeled video as a mid‑training phase for a pretrained language model. By encoding video frames into visual tokens and training the model to predict the next token, the authors mid‑train Qwen3‑1.7B on YT‑Temporal‑1B clips. The resulting model outperforms a non‑mid‑trained counterpart by 2.9 points on video benchmarks and 5.1 points on image benchmarks, while maintaining comparable text performance.
By Jaedong Hwang, Xiaoqian Shen, Ernie Chang, Changsheng Zhao, Chong Zhou, Saksham Suri, Qi Qian, Zechun Liu, Lemeng Wu, Qinsi Wang, Raghuraman Krishnamoorthi, Wei Wen
Flow-CLIP introduces a method to improve compositional image-text matching by refining global image-text alignment with image-conditioned changes in token–patch evidence. It combines Global Compatibility Calibration (GCC) for sentence-level similarity, Image-Conditioned Evidence Refinement (ICER) to incorporate visual patch context, and Cross-Image Margin Ranking (CIMR) to enhance ranking margins when images support correct descriptions. Experiments demonstrate that Flow-CLIP outperforms pretrained CLIP and other global alignment approaches on binding accuracy and reduces reliance on language priors.
By Minghong Wei, Enguang Zhu, Pu Cao, Lu Yang, Qing Song
arXiv:2602. 21397v2 Announce Type: replace-cross Abstract: Prompt learning has become a dominant paradigm for adapting vision-language models (VLMs) such as CLIP to downstream tasks without modifying pretrained weights.
By Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh, Ramtin Pedarsani
arXiv:2607. 23271v1 Announce Type: cross Abstract: Contrastive vision-language models such as CLIP map semantically opposite phrases (e.
By Chen-Yi Lu, Yueh-Shao Chen, Somali Chaterji
MLLMCLIP introduces a heterogeneous distillation framework that transfers multimodal knowledge from a generative Multimodal Large Language Model (MLLM) teacher directly into a discriminative CLIP student, eliminating the need for synthetic hard negatives. The method uses an attention-based per-layer token selection and a CKA-based distillation loss to bridge architectural differences between the two models. As a result, MLLMCLIP achieves state‑of‑the‑art compositional accuracy and improves zero‑shot classification and image‑text retrieval performance.
By Jongsuk Kim, Qiyu Wu, Zhuoyuan Mao, Hiromi Wakaki, Junmo Kim, Yuki Mitsufuji
Contrastive Language-Image Pre-training (CLIP) has been shown to have limitations in its fine-grained dense feature representation, due to its pre-training focusing on matching the whole image to a text description. Considering the large data and computational burden in pre-training a vision-language model from scratch, a series of works aim to enhance the fine-grained ability of CLIP through a fine-tuning scheme.