arXiv Machine Learning July 22, 2026 By Micha{\l} Paw{\l}owicz

KALE: Kernel Alignment with Loss Equilibration for Stable CLIP-DINOv2 Alignment at Web Scale

Read the original on arXiv Machine Learning →

arXiv:2607. 18885v1 Announce Type: new Abstract: Kernel-based alignment of CLIP toward a vision centric teacher such as DINOv2 (KUEA) improves CLIP's visual representations while preserving text-encoder compatibility, using a fixed trade-off weight tuned on curated ImageNet-1K.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computer Vision

Sep 10

UOT-Gap: A Variational Principle for the Modality Gap in Vision-Language Models via Unbalanced Optimal Transport

arXiv:2609.10224v1 Announce Type: new Abstract: Vision-language models such as CLIP embed images and text in a shared space, where modality-specific distributions often remain separated. Existing acc...

By Zonglin Yang, Huilan Ma, Xudan Zheng, Yuejun Xie

llms rag multimodal safety

arXiv AI

Sep 10

Bigger Text Encoders Can Hurt CLIP Zero-Shot Performance

arXiv:2609.05730v1 Announce Type: cross Abstract: Contrastive Language-Image Pretraining (CLIP) is a building block of many machine learning applications. Scaling laws have guided resource allocation...

By Samir Char, Carles Domingo-Enrich, Randall Balestriero

rag multimodal safety

arXiv Machine Learning

Jul 28

Shift-Aware Calibration for Fine-Tuned CLIP: Leveraging Image-Text Alignment

arXiv:2501. 19060v4 Announce Type: replace-cross Abstract: Vision-language models (VLMs), such as CLIP, adapt effectively to downstream tasks through prompt tuning, but fine-tuning can misalign predictive confidence and accuracy, particularly on unseen classes.

By Song-Lin Lv, Yu-Yang Chen, Zhi Zhou, Lan-Zhe Guo

llms fine-tuning multimodal safety

arXiv AI

Sep 10

TDDN: Text-aligned Diffused DINO Network for Puzzle Understanding

arXiv:2609.07937v1 Announce Type: cross Abstract: Structured visual reasoning, such as image puzzles, demands fine-grained visual perception, an ability current Vision Language Models (VLMs) lack. VL...

By Harsha Patnala, Debopriyo Banerjee, Ayush Sunil Munot, Somak Aditya

llms computer-vision nlp multimodal benchmarks safety

arXiv AI

Jul 21

A${}^2$BM: Alignment-Aware Bridge Matching for Image-to-Image Translation

arXiv:2607. 16294v1 Announce Type: cross Abstract: Paired image-to-image translation underpins a wide range of computer vision tasks, including image editing, sensor translation, and domain adaptation.

By Aimi Okabayashi (UBS Vannes), Georges Le Bellier (LIP, CEDRIC - VERTIGO), Nicolas Audebert (LaSTIG, IGN, CEDRIC - VERTIGO), Charlotte Pelletier (OBELIX), Thomas Corpetti (LETG - Rennes), Nicolas Courty (OBELIX)

llms diffusion computer-vision fine-tuning safety

arXiv Computer Vision

Sep 11

Task Alignment: A Simple Proxy for Practical Model Merging Across Diverse Vision Tasks

The paper introduces the task alignment proxy, a method that accelerates hyperparameter selection for merging models fine‑tuned on diverse vision tasks. It addresses the challenge of training heterogeneous decoders, which makes traditional downstream performance evaluation costly. By using the proxy, the authors demonstrate that model merging can be applied efficiently to multi‑task vision models beyond CLIP‑based classification.

By Pau de Jorge, C\'esar Roberto de Souza, Bj\"orn Michele, Mert B\"ulent Sar{\i}y{\i}ld{\i}z, Philippe Weinzaepfel, Florent Perronnin, Diane Larlus, Yannis Kalantidis

computer-vision fine-tuning safety

One email a morning, machine-written

Related stories

UOT-Gap: A Variational Principle for the Modality Gap in Vision-Language Models via Unbalanced Optimal Transport

Bigger Text Encoders Can Hurt CLIP Zero-Shot Performance

Shift-Aware Calibration for Fine-Tuned CLIP: Leveraging Image-Text Alignment

TDDN: Text-aligned Diffused DINO Network for Puzzle Understanding

A${}^2$BM: Alignment-Aware Bridge Matching for Image-to-Image Translation

Task Alignment: A Simple Proxy for Practical Model Merging Across Diverse Vision Tasks