arXiv Machine Learning July 28, 2026 By Song-Lin Lv, Yu-Yang Chen, Zhi Zhou, Lan-Zhe Guo

Shift-Aware Calibration for Fine-Tuned CLIP: Leveraging Image-Text Alignment

Read the original on arXiv Machine Learning →

arXiv:2501. 19060v4 Announce Type: replace-cross Abstract: Vision-language models (VLMs), such as CLIP, adapt effectively to downstream tasks through prompt tuning, but fine-tuning can misalign predictive confidence and accuracy, particularly on unseen classes.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 2

MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation

arXiv:2602. 21397v2 Announce Type: replace-cross Abstract: Prompt learning has become a dominant paradigm for adapting vision-language models (VLMs) such as CLIP to downstream tasks without modifying pretrained weights.

By Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh, Ramtin Pedarsani

llms rag multimodal benchmarks safety

arXiv Machine Learning

Jul 22

KALE: Kernel Alignment with Loss Equilibration for Stable CLIP-DINOv2 Alignment at Web Scale

arXiv:2607. 18885v1 Announce Type: new Abstract: Kernel-based alignment of CLIP toward a vision centric teacher such as DINOv2 (KUEA) improves CLIP's visual representations while preserving text-encoder compatibility, using a fixed trade-off weight tuned on curated ImageNet-1K.

By Micha{\l} Paw{\l}owicz

multimodal safety

arXiv Machine Learning

Jun 15

What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

arXiv:2606. 14299v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) such as CLIP have become a standard backbone for open-vocabulary recognition, yet their zero-shot predictions remain vulnerable to distribution shifts encountered at deployment.

By Jiazhen Huang, Xiao Chen, Zhiming Liu, Yaru Sun, Jingyan Jiang, Zhi Wang

llms multimodal benchmarks

arXiv AI

Jun 16

Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment

arXiv:2606. 16799v1 Announce Type: cross Abstract: Existing vision-language model (VLM)-based AI-generated image quality assessment (AIGIQA) methods suffer from a fundamental semantic-distortion dimensional conflict: monolithic representations optimized for semantic discrimination inherently entangle compositional understanding with low-level perceptual sensitivity, rendering them blind to fine-grained quality degradations.

By Zijie Meng

llms efficiency multimodal benchmarks safety

arXiv AI

Jun 17

Plug-and-Adapt: Multimodal Coreference Resolution at First Sight with a Pretrained Alignment Model

arXiv:2606. 17950v1 Announce Type: cross Abstract: Visual information helps resolve ambiguity in coreference resolution, leading to notable performance gains.

By Jinghan Wu, Jing Li, Ivor W. Tsang, Xuetao Zhang

multimodal benchmarks safety

arXiv AI

Jun 2

Detect Before You Leap: Mirage Detection in Vision-Language Models

arXiv:2606. 00435v1 Announce Type: cross Abstract: Vision-language models (VLMs) can produce confident visual answers even when the required visual evidence is missing, blank, or unrelated to the question.

By Sayeed Shafayet Chowdhury, Md. Shaown Miah

llms rag nlp multimodal safety

Related stories

MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation

KALE: Kernel Alignment with Loss Equilibration for Stable CLIP-DINOv2 Alignment at Web Scale

What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment

Plug-and-Adapt: Multimodal Coreference Resolution at First Sight with a Pretrained Alignment Model

Detect Before You Leap: Mirage Detection in Vision-Language Models