arXiv:2609.36648v1 Announce Type: new Abstract: Vision-language pre-training has reshaped image clustering, giving rise to language-assisted image clustering (LaIC), which leverages textual semantics...
arXiv:2609.10224v1 Announce Type: new Abstract: Vision-language models such as CLIP embed images and text in a shared space, where modality-specific distributions often remain separated. Existing acc...
CIM-Embed is a framework that learns instruction‑conditioned multimodal embeddings for clustering. It uses supervised contrastive learning with same‑group positives and negatives from other groups, and handles tasks with varying group counts via heterogeneous structured batches. At inference, the model maps an object’s image, text, and instruction to an embedding, and HDBSCAN clusters these embeddings without requiring a pre‑specified cluster count. The authors introduce ProductMIC, a benchmark of 5,373 Amazon products and 449 grouping tasks, and report a mean per‑task V‑measure of 95.01, outperforming the best general‑purpose baseline.
UVU is a vision-language unified autoregressive framework that integrates visual supervision directly into the pre-training stage of multimodal large language models. By using continuous visual encoding and a large-scale iterative hierarchical clustering algorithm to build a pixel-level visual codebook, UVU enables lossless representation of visual inputs and autoregressive generation of pixel-level image tokens alongside textual tokens. This approach synergizes pixel-level visual perception with semantic-level visual understanding, allowing models to internalize visual reconstruction capabilities and improve multimodal understanding performance.
arXiv:2608. 19285v1 Announce Type: cross Abstract: Recent Visual-Language Models (VLMs) have enhanced the capabilities of pre-trained LLMs by adding vision tokens alongside text, with approaches like LLaVA showing impressive results.
arXiv:2606. 24716v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence.
arXiv:2607. 03994v1 Announce Type: cross Abstract: Modern language models generally represent text as sequences of discrete token embeddings, an assumption deeply rooted in current practice but rarely questioned.
arXiv:2606. 28551v1 Announce Type: cross Abstract: Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies.
arXiv:2608.30263v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) incur substantial inference costs due to their long and highly redundant visual-token sequences. Diversity-based...
The paper investigates whether pretrained image models can generalize to unseen datasets by clustering their embeddings. Using encoders trained only on ImageNet‑1k, both supervised and self‑supervised, the authors evaluate clustering performance on out‑of‑domain images. They find that supervised encoders perform better within the training domain, while self‑supervised encoders excel far outside it, and that fine‑tuning self‑supervised models reverses this trend. Additionally, the study shows that the silhouette score in UMAP‑reduced space correlates strongly with clustering accuracy, offering a proxy metric when labels are unavailable.
arXiv:2605.00809v3 Announce Type: replace Abstract: In this paper, we present \textbf{Gen}erative \textbf{L}anguage-\textbf{I}mage \textbf{P}re-training (GenLIP), a minimalist generative pretraining...
arXiv:2512. 08240v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs.