arXiv AI
Jun 30
Multimodal Representation Alignment for Cross-modal Information Retrieval
arXiv:2506. 08774v2 Announce Type: replace-cross Abstract: Different machine learning models can represent the same underlying concept in different ways.
COMBINER: Composed Image Retrieval Guided by Attribute-based Neighbor Relations
Composed Image Retrieval (CIR) represents a challenging retrieval task that targets locating specific images through multimodal inputs. Despite recent progress in CIR techniques, prior approaches often overlook cases where images appear visually alike yet differ in attributes, potentially undermining both multimodal feature fusion and similarity modeling.
Related stories
arXiv AI
Jul 7
SpectralGCD: Spectral Concept Selection and Cross-modal Representation Learning for Generalized Category Discovery
arXiv:2602. 17395v2 Announce Type: replace-cross Abstract: Generalized Category Discovery (GCD) aims to identify novel categories in unlabeled data while leveraging a small labeled subset of known classes.
arXiv AI
Jul 28
Controlling Embedding Spaces with Text-Conditioned Transformations
arXiv:2607. 22919v1 Announce Type: cross Abstract: Multimodal embedding spaces in models like CLIP enable powerful capabilities such as semantic similarity retrieval and cross-modal zero-shot classification.
arXiv AI
5d ago
Generative Universal Multimodal Retrieval with Dual-role Identifiers
arXiv:2608. 12987v1 Announce Type: cross Abstract: Generative information retrieval (GIR) has emerged as a compelling alternative to the conventional index-retrieve-then-rank retrieval pipeline by training a generator to produce the identifiers of relevant items directly.
arXiv Machine Learning
Jun 30
Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders
arXiv:2606. 29888v1 Announce Type: new Abstract: Vision-language models map images and text into a joint embedding space.
arXiv AI
Jun 16
Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings
arXiv:2606. 15134v1 Announce Type: cross Abstract: Vision encoders for retrieval are typically trained with class-label supervision: each training pair reduces to a scalar that uniformly pushes the embedding apart or pulls it together, as if every visual attribute either differed or matched.
arXiv AI
Jul 21
DCSCR: A Class-Specific Collaborative Representation based Network for Image Set Classification
arXiv:2508. 12745v2 Announce Type: replace-cross Abstract: Image set classification (ISC), which can be viewed as a task of comparing similarities between sets consisting of unordered heterogeneous images with variable quantities and qualities, has attracted growing research attention in recent years.
Hugging Face Trending Papers
Jul 30
FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrieval
Due to their strong generalizable multimodal processing and reasoning capabilities, Multimodal Large Language Models (MLLMs) have demonstrated significant potential as universal image retrievers, effectively addressing diverse real-world image retrieval tasks. Nevertheless, pioneering studies, while promising, overlook the potential of fine-grained context modeling and disentangled fine-tuning objectives in enhancing MLLMs' retrieval performance, particularly for complex tasks such as long-text-to-image retrieval, visual dialog retrieval, and composed image retrieval (CIR).
arXiv Machine Learning
1d ago
FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens
arXiv:2506. 03096v2 Announce Type: replace-cross Abstract: Contrastive language-image pre-training aligns features of text-image pairs in a common latent space via distinct encoders for each modality.
arXiv Machine Learning
Jun 4
RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities
arXiv:2606. 05109v1 Announce Type: new Abstract: To leverage the full potential of multimodal data, we need representations that go beyond the state-of-the-art alignment and fusion approaches and exploit all cross-modal interactions without sacrificing modality-specific information.
arXiv AI
Jul 3
SEPS: Semantic-enhanced Patch Slimming Framework for fine-grained cross-modal alignment
arXiv:2511. 01390v2 Announce Type: replace-cross Abstract: Fine-grained cross-modal alignment aims to establish precise local correspondences between vision and language, forming a cornerstone for visual question answering and related multimodal applications.
arXiv AI
Jun 16
MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios
arXiv:2606. 14747v1 Announce Type: cross Abstract: Recent advancements have significantly expanded the theoretical context windows of Multimodal Embedding Models (MEMs).