← Back to all news
arXiv AI By Yinghua Xu, Pengyang Xie, Yanxiang Zhou, Chuanzhi Tong, Xiao Xiao, Chaoqun Feng, Xiaoqing Wang

CIM-Embed: Learning Instruction-Conditioned Multimodal Embeddings for Clustering

Read the original on arXiv AI →

CIM-Embed is a framework that learns instruction‑conditioned multimodal embeddings for clustering. It uses supervised contrastive learning with same‑group positives and negatives from other groups, and handles tasks with varying group counts via heterogeneous structured batches. At inference, the model maps an object’s image, text, and instruction to an embedding, and HDBSCAN clusters these embeddings without requiring a pre‑specified cluster count. The authors introduce ProductMIC, a benchmark of 5,373 Amazon products and 449 grouping tasks, and report a mean per‑task V‑measure of 95.01, outperforming the best general‑purpose baseline.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 26

HMGCLIP: Heterogeneous Multi-Granularity Contrastive Learning for E-commerce Representation Learning

HMGCLIP is a unified multimodal embedding framework that uses a heterogeneous hypergraph to capture both fine‑grained and coarse‑grained product attributes. By mining structure‑aware hard negatives and aligning multi‑granular semantics at relation and hyperedge levels, it enables a dual‑granularity inference mechanism that dynamically fuses attribute evidence. Experiments on a new fine‑grained e‑commerce dataset and the public MAVE benchmark show that HMGCLIP outperforms strong multimodal encoders, MLLMs, and e‑commerce baselines.

By Qiuyu Zhu, Yi Gao, Zhichao Wan, Mingyang Ma
llmsragmultimodalbenchmarks
More like this →
arXiv AI
Sep 18

Lens: Bringing the Right Semantic Perspective into Focus for Training-Free Multimodal Representation Learning

The paper introduces Lens, a training‑free framework that aligns multimodal representations with the semantic perspective required by downstream tasks. Lens uses a task‑specific readout phrase to anchor the perspective and then aggregates token states after the full input, ensuring the extracted representation reflects task‑conditioned evidence integration rather than generic salient content. The method achieves a Precision@1 of 63.9 across 36 MMEB datasets, outperforming the nearest training‑free baseline by 10.2 points.

By Xinran Liu, Shouqian Shi, Yixian Chen, Ruizhi Chen, Xin-Wei Yao, Sheng Zhong
llmsragmultimodal
More like this →