arXiv Computer Vision
Aug 27

AdaptiveEmbed: Sample-Adaptive Multi-Vector Representation for Multimodal Retrieval

The paper introduces AdaptiveEmbed, a framework that learns sample‑adaptive multi‑vector representations for multimodal retrieval. It proposes the SAMVR setting, where each sample receives a content‑adaptive embedding set whose size is determined by its retrieval utility. The method uses Multi‑Group Contrastive Learning with set‑to‑set similarity and Utility Policy Optimization to allocate representation capacity, achieving better performance across image, text, video, and audio benchmarks.

By Xinze Liu, Lei Yang, Dayan Wu, Hengjie Zhu, Zihao Zhang, Hanqi Wu, Tianzhu Hu, Peng Fu, Zheng Lin, Weiping Wang