arXiv Machine Learning
Jul 21
The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric
arXiv:2607. 18237v1 Announce Type: cross Abstract: Human visual similarity judgments are context-dependent.
CLAY: Conditional Visual Similarity Modulation in Vision-Language Embedding Space
arXiv:2604. 11539v2 Announce Type: replace-cross Abstract: Human perception of visual similarity is inherently adaptive and subjective, depending on the users' interests and focus.
Related stories
arXiv AI
Jun 8
TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignment
arXiv:2606. 07451v1 Announce Type: cross Abstract: Vision-language models such as CLIP are highly useful for diverse tasks due to their shared image-text embedding space.
arXiv Machine Learning
Jul 28
Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models
arXiv:2607. 23052v1 Announce Type: cross Abstract: Dual-encoder vision-language models (VLMs) expose a similarity interface that enables zero-shot retrieval but fails compositional constraints: queries like "umbrella and no person" retrieve images containing both, even when concept detection is reliable.
arXiv Computation and Language
Sep 15
Human-Grounded Calibration for Long-Text Image-Text Congruence in Vision-Language Models
arXiv:2609.15640v1 Announce Type: cross Abstract: Long-text image--text congruence scoring is increasingly important for vision-language systems that must evaluate whether detailed textual descriptio...
arXiv AI
Jun 30
Multimodal Representation Alignment for Cross-modal Information Retrieval
arXiv:2506. 08774v2 Announce Type: replace-cross Abstract: Different machine learning models can represent the same underlying concept in different ways.
arXiv AI
Jun 16
Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings
arXiv:2606. 15134v1 Announce Type: cross Abstract: Vision encoders for retrieval are typically trained with class-label supervision: each training pair reduces to a scalar that uniformly pushes the embedding apart or pulls it together, as if every visual attribute either differed or matched.
arXiv AI
Jul 28
Controlling Embedding Spaces with Text-Conditioned Transformations
arXiv:2607. 22919v1 Announce Type: cross Abstract: Multimodal embedding spaces in models like CLIP enable powerful capabilities such as semantic similarity retrieval and cross-modal zero-shot classification.
arXiv Computer Vision
Sep 3
MARS: What Retrieval Signals Are Hidden in Multimodal Large Language Models for Text-Video Retrieval?
MARS introduces a multi‑layer, multi‑slot embedding framework for text‑video retrieval that constructs adaptive representation slots by combining hidden states from different decoder layers. By comparing corresponding text and video slots and aggregating their similarities, MARS captures fine‑grained cues that single‑token embeddings miss. A hard‑negative‑aware slot specialization objective further encourages slots to focus on discriminative matching cues, leading to state‑of‑the‑art results on four benchmarks.
arXiv Machine Learning
Jun 2
Density-Aware Translation of Spurious Correlations in Zero-Shot VLMs
arXiv:2606. 01710v1 Announce Type: cross Abstract: Vision-Language models (VLMs), such as CLIP, achieve powerful zero-shot classification.
arXiv Machine Learning
Aug 5
Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding
arXiv:2608. 03826v1 Announce Type: cross Abstract: Geospatial and urban applications increasingly require models to compare heterogeneous evidence across street-view imagery, remote-sensing observations, text descriptions, region proposals, and temporal change cues.
Hugging Face Trending Papers
Aug 4
Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding
Geospatial and urban applications increasingly require models to compare heterogeneous evidence across street-view imagery, remote-sensing observations, text descriptions, region proposals, and temporal change cues. However, existing multimodal embedding models and benchmarks are still largely designed and evaluated around general-purpose image-text matching, leaving unclear whether unified embedding space can support heterogeneous geospatial tasks involving spatial relationships, fine-grained semantics, and temporal changes.
arXiv AI
Sep 3
ViSAR: Training-Free Adaptive-$k$ Retrieval for Visual Document Question Answering
ViSAR is a training‑free, adaptive‑k retrieval method for Visual Document Question Answering that operates directly in the embedding space to build a query‑conditioned page‑level similarity matrix. By dynamically selecting the number of pages to retrieve based on query relevance, ViSAR reduces Retrieval‑Augmented Generation latency by up to 58.7% while maintaining or improving answer accuracy across multiple encoders and Large Vision‑Language Models. The structure of the similarity matrix also correlates with answer accuracy, indicating potential for retrieval quality‑aware document understanding.