The paper surveys the state of embedding models across different modalities, noting that despite shared mathematical foundations, research remains fragmented. It argues that recent advances—contrastive learning, Transformer architectures, and a shift toward finer granularity—are driving a convergence toward common principles, while modality-specific challenges are still underexplored. The authors propose a framework of three capabilities—coverage, comparability, and compositionality—and outline three unification paths: transformation, alignment, and joint embedding, with the latter advocated as the ultimate goal.
Universal multimodal embeddings are becoming a core component of modern AI systems, enabling heterogeneous content to be represented in a shared space for applications such as retrieval, recommendatio...
Syn-Omni introduces a structured approach to omnimodal embeddings by separating universal semantics from modality-specific features through Orthogonal Modality-Expert LoRA (OME-LoRA). It further employs Progressive Synergy Routing (PSR) to allow modality experts to first learn modality-specific priors and then gradually collaborate across modalities. Across 81 tasks involving image, video, audio, and audiovisual data, Syn-Omni outperforms existing omnimodal baselines, highlighting the benefits of structured specialization and progressive cross-modal collaboration.
MoEMB introduces a mixture‑of‑experts (MoE) approach to scale universal multimodal embeddings (UME) without increasing the size of the output vector or relying on autoregressive decoding. By expanding encoder capacity along the expert axis, MoEMB achieves state‑of‑the‑art performance on MMEB‑V2 and MRMR benchmarks with only 3 B active parameters, outperforming TTE‑based methods that use more than four times as many active parameters and require significantly more compute. The paper also presents the first comprehensive study of adaptive computation for MoE‑based embeddings, exploring training‑time and inference‑time strategies to further improve efficiency for large‑scale retrieval and recommendation systems.
arXiv:2608. 05980v1 Announce Type: new Abstract: We investigate whether simple transformations can translate representations across heterogeneous text embedding models.
arXiv:2608.24053v1 Announce Type: new Abstract: Universal multimodal embeddings are becoming a core component of modern AI systems, enabling heterogeneous content to be represented in a shared space...
arXiv:2603. 14405v2 Announce Type: replace-cross Abstract: Biological multimodal large language models (MLLMs) have emerged as powerful foundation models for scientific discovery.
The paper introduces Biomedica, an open-source dataset sourced from PubMed Central that includes over 6 million scientific articles and 24 million image‑text pairs, along with 27 metadata fields and expert human annotations. To facilitate use, the authors provide scalable streaming and search APIs via a web server. They demonstrate the dataset’s value by training embedding models, chat‑style models, and retrieval‑augmented chat agents, all of which outperform previous open systems in their categories.
Multimodal fusion learning (MFL) has shown great potential in the medical domain, where we are faced with disparate data modalities such as imaging, clinical records, and omics. However, existing MFL strategies face several major challenges.
arXiv:2603. 01471v3 Announce Type: replace-cross Abstract: Multimodal embedding models, rooted in multimodal large language models (MLLMs), have yielded significant performance improvements across diverse tasks such as retrieval and classification.
arXiv:2603. 17450v2 Announce Type: replace-cross Abstract: Sequential Recommendation (SR) in multimodal settings typically relies on small frozen pretrained encoders, which limits semantic capacity and prevents Collaborative Filtering (CF) signals from being fully integrated into item representations.
Bridge-LP is a new framework for link prediction on text‑attributed graphs that combines Graph Neural Networks and Language Models. It introduces three design components: jumping connections for robust language model training, compressed embeddings to encode neighborhood information, and a cache embedding strategy to reduce computational cost. Experiments on nine datasets show that Bridge‑LP outperforms 40 existing models, achieving up to a 43.06% improvement in Mean Reciprocal Rank.
CORE improves compositional reasoning in multimodal language models by distilling a cross‑attentive reranker’s fine‑grained judgments into the embedding model. It generates candidate lists across five compositional matching levels and trains with a Rank‑KL objective to replicate the reranker’s ranking. Experiments on COLA, SUGARCREPE++, and NEGBENCH show CORE‑RERANKER‑8B outperforms Jina‑Reranker by 10.7 points, while CORE‑EMBED‑8B achieves the best overall average among evaluated embeddings, with gains also transferring to the MCMR benchmark without harming COCO or Flickr30K retrieval.