arXiv AI October 12, 2026 By Julien Kordes, Reuben Dorent

Representation Alignment from a Few Paired Data via Distilled Optimal Transport

The paper introduces a method for aligning two independently pretrained unimodal encoders using only a small set of paired examples (≈100–500) while leveraging abundant unpaired data. It represents each modality by similarities to the paired anchors, matches the resulting point clouds via a single entropic optimal transport problem that mitigates hubness, and distills the transport plan into lightweight encoders for inductive correspondence. Experiments on image‑text and molecule‑text datasets (Flickr30k, MS‑COCO, ChEBI‑20) show that this approach outperforms existing methods under the same anchor budget.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 30

SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport

arXiv:2602. 23353v2 Announce Type: replace-cross Abstract: The Platonic Representation Hypothesis posits that neural networks trained on different modalities converge toward a shared statistical model of the world.

By Simon Roschmann, Paul Krzakala, Sonia Mazelet, Quentin Bouniot, Zeynep Akata

llms rag multimodal safety

arXiv Machine Learning

Aug 6

Multimodal Alignment Through Joint Kernel Entropic Gromov--Wasserstein Optimal Transport

arXiv:2608. 04234v1 Announce Type: cross Abstract: We study the problem of aligning data from multiple modalities into a shared representation space, focusing on settings where strong pretrained unimodal encoders are available but cross-modal paired data are scarce.

By Yixuan Florence Wu, Yilun Zhu, Naichen Shi

rag diffusion multimodal safety

arXiv AI

Jun 30

Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

arXiv:2606. 29464v1 Announce Type: cross Abstract: Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision-language models under strict data and compute budgets.

By Jongoh Jeong, Sun-Kyung Lee, Kuk-Jin Yoon

llms rag efficiency multimodal benchmarks safety

arXiv Computer Vision

Sep 22

BindCLIP: One Balanced Coupling For Compositional Vision Language Scoring

arXiv:2609.23717v1 Announce Type: new Abstract: Global vision--language similarities compress an image and a caption into one vector, preserving semantics but not which word corresponds to which regi...

By Liuyang Song, Yi Zhang, Zhongyi Deng, Daqian Yang, Hongbo Zhang

benchmarks

arXiv AI

Oct 5

Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis

arXiv:2610.02718v1 Announce Type: cross Abstract: Vision-language models such as CLIP exhibit strong semantic generalization, but remain limited in fine-grained visual perception. A recent work named...

By Peilin Yang, Xiaoyu Liu, Jian Sun, Qinghua Tao

llms rag fine-tuning multimodal safety

Hugging Face Trending Papers

Sep 20

BindCLIP: One Balanced Coupling For Compositional Vision Language Scoring

Global vision--language similarities compress an image and a caption into one vector, preserving semantics but not which word corresponds to which region or how those regions are arranged; a model can...

benchmarks

One email a morning, machine-written

Related stories

SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport

Multimodal Alignment Through Joint Kernel Entropic Gromov--Wasserstein Optimal Transport

Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

BindCLIP: One Balanced Coupling For Compositional Vision Language Scoring

Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis

BindCLIP: One Balanced Coupling For Compositional Vision Language Scoring