arXiv AI
Image Clustering with Vision-Language Model Embeddings
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computer Vision
Sep 30
VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training
arXiv:2609.36648v1 Announce Type: new Abstract: Vision-language pre-training has reshaped image clustering, giving rise to language-assisted image clustering (LaIC), which leverages textual semantics...
arXiv Computer Vision
Sep 10
UOT-Gap: A Variational Principle for the Modality Gap in Vision-Language Models via Unbalanced Optimal Transport
arXiv:2609.10224v1 Announce Type: new Abstract: Vision-language models such as CLIP embed images and text in a shared space, where modality-specific distributions often remain separated. Existing acc...
arXiv AI
Oct 16
CIM-Embed: Learning Instruction-Conditioned Multimodal Embeddings for Clustering
CIM-Embed is a framework that learns instruction‑conditioned multimodal embeddings for clustering. It uses supervised contrastive learning with same‑group positives and negatives from other groups, and handles tasks with varying group counts via heterogeneous structured batches. At inference, the model maps an object’s image, text, and instruction to an embedding, and HDBSCAN clusters these embeddings without requiring a pre‑specified cluster count. The authors introduce ProductMIC, a benchmark of 5,373 Amazon products and 449 grouping tasks, and report a mean per‑task V‑measure of 95.01, outperforming the best general‑purpose baseline.
arXiv Computer Vision
Sep 24
UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm
UVU is a vision-language unified autoregressive framework that integrates visual supervision directly into the pre-training stage of multimodal large language models. By using continuous visual encoding and a large-scale iterative hierarchical clustering algorithm to build a pixel-level visual codebook, UVU enables lossless representation of visual inputs and autoregressive generation of pixel-level image tokens alongside textual tokens. This approach synergizes pixel-level visual perception with semantic-level visual understanding, allowing models to internalize visual reconstruction capabilities and improve multimodal understanding performance.
arXiv Machine Learning
Aug 21
Clustering and Token Denoising for Faster and More Robust VLMs
arXiv:2608. 19285v1 Announce Type: cross Abstract: Recent Visual-Language Models (VLMs) have enhanced the capabilities of pre-trained LLMs by adding vision tokens alongside text, with approaches like LLaVA showing impressive results.
arXiv AI
Jun 24
Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
arXiv:2606. 24716v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence.