arXiv Computer Vision

RankAlign: Vision-Language Representation Alignment via Rank Transformation

RankAlign introduces a rank-based normalization technique for aligning vision and language representations without cross-modal supervision. By converting within-modality similarity kernels into ordinal relational representations, it preserves the relative ordering of similarities while remaining invariant to strictly increasing reparameterizations. The method offers theoretical guarantees for exact and partial ordinal agreement and shows improved, more stable class-level correspondence recovery across various frozen encoders and datasets, even under label corruption and domain shift.

arXiv AI
Jun 24

Mind the Heads: Topological Representation Alignment for Multimodal LLMs

arXiv:2606. 23885v1 Announce Type: cross Abstract: Representation alignment has emerged as an effective approach to improve Multimodal Large Language Models (MLLMs) by regularizing their internal representations toward those of an external vision encoder.

By Davide Caffagni, Alberto Compagnoni, Federico Melis, Sara Sarto, Pier Luigi Dovesi, Mark Granroth-Wilding, Marcella Cornia, Lorenzo Baraldi
arXiv AI
Jul 7

Multi-Way Representation Alignment

arXiv:2602. 06205v2 Announce Type: replace-cross Abstract: The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces.

By Akshit Achara, Tatiana Gaintseva, Mateo Mahaut, Pritish Chakraborty, Viktor Stenby Johansson, Melih Barsbey, Emanuele Rodol\`a, Donato Crisostomi
arXiv Computer Vision
Oct 26

Separating Angular Alignment from Radial Ordering in Hyperbolic Vision-Language Models

The paper introduces AURORA, a hyperbolic vision‑language framework that separates angular alignment from radial ordering. Angular alignment captures semantic similarity while a novel Radial‑Angular Order Cone (RAOC) models hierarchical entailment based on parent‑child radial gaps. The approach includes norm regularization to prevent radial degeneration and is evaluated on large image‑text datasets, outperforming both Euclidean and hyperbolic baselines in zero‑shot classification, retrieval, hierarchical reasoning, and compositional understanding.

By Mingyang Gao, Yuwei Wu, Shiyu Luo, Wei Wu, Xiaomeng Fan, Qihao Lv, Haoyu Zhang, Rui Gao, Bingzhen Liu, Zhi Gao, Che Sun
Hugging Face Trending Papers
Jun 3

Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain

Vision-Language Models (VLMs) struggle when applied to medical image-text data, yet the tools available to diagnose this failure remain limited. Existing representation alignment metrics are symmetric, collapsing both modalities into a single score and hiding which modality drives cross-modal degradation.

arXiv AI
Sep 30

One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models

The paper investigates how the modality gap— the separation between image and text representations in contrastive vision‑language models—affects different downstream tasks. By showing that a single dominant direction accounts for most of the image‑text mean separation, the authors explain why reducing or removing this gap can improve zero‑shot classification, degrade retrieval, or restore performance depending on the task. The study provides a geometric framework that clarifies when and why gap interventions should be applied in vision‑language systems.

By Aditya Sharma, Divya Saxena