arXiv Computer Vision October 2, 2026 By Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly, Ivan Laptev, Hisham Cholakkal

Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation

Read the original on arXiv Computer Vision →

Omni-Embed-Mini is a 0.9B‑parameter model that embeds text, speech, audio, images, video, and visually‑rich documents into a single shared cosine space without updating any text‑side parameters. It uses a dense cascaded caption as a teacher signal, allowing the teacher and student to share identical backbone weights and requiring only lightweight projectors and phased LoRA adapters for alignment. The model achieves strong text retrieval performance (49.57 nDCG@10 on MTEB‑v2 BEIR‑8) while extending to five additional modalities and is significantly smaller than other open omni‑modal embedders.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision

Sep 10

UOT-Gap: A Variational Principle for the Modality Gap in Vision-Language Models via Unbalanced Optimal Transport

arXiv:2609.10224v1 Announce Type: new Abstract: Vision-language models such as CLIP embed images and text in a shared space, where modality-specific distributions often remain separated. Existing acc...

By Zonglin Yang, Huilan Ma, Xudan Zheng, Yuejun Xie

llms rag multimodal safety

arXiv AI

Jun 9

Conan-embedding-v3: Fusing Modality-Specific Models for Omni-Modal Embedding

arXiv:2606. 09331v1 Announce Type: cross Abstract: Omni-modal retrieval promises a single embedding space for text, image, video, document, and audio inputs, but building such a unified retriever is difficult since these modalities differ in data distribution, architecture, and optimization dynamics.

By Shiyu Li, Zhiyuan Hu, Yifan Wang, Peiming Li, Zheng Wei, Yang Tang

rag fine-tuning multimodal

arXiv Computer Vision

Sep 15

Hypergraph-Regularized Gramian Volumes for Multimodal Retrieval

arXiv:2609.15320v1 Announce Type: new Abstract: Volume-based multimodal retrieval jointly scores a text query with a candidate's video, audio, and subtitle embeddings. While this approach captures hi...

By Anindya Nag, Ambuj Mehrish, Sebastiano Vascon

rag multimodal benchmarks safety

arXiv Computer Vision

Aug 26

WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report

arXiv:2608.24053v1 Announce Type: new Abstract: Universal multimodal embeddings are becoming a core component of modern AI systems, enabling heterogeneous content to be represented in a shared space...

By Junjie Zhou, Ke Mei, Lei Li, Tianyi Wang, Fengyun Rao, Jing Lyu

rag agents multimodal benchmarks safety

arXiv AI

Aug 26

OmniJudge or OmniBias? Diagnosing Multimodal Judges through Balanced, Decoupled Lenses

The paper introduces D3-Omni, a balanced and decoupled benchmark designed to diagnose fine‑grained multimodal understanding in OmniJudges that evaluate text‑to‑image, text‑to‑video, and text‑to‑speech generation. D3-Omni covers 53 orthogonal binary dimensions across 10,671 samples, using fixed positive seeds and controlled prompt rewriting to generate negatives, thereby ensuring each error can be attributed to a single capability. The benchmark’s dual‑balanced, decoupled, and dynamic design achieves near 1:1 per‑dimension parity and a uniform total‑score distribution, revealing that strong OmniJudges often miss modality‑related failures and treat distinct attributes as a single decision, masking systematic blind spots.

By Guangzheng Hu, Ziyue Jiang, Weixu Qiao, Lixin Zhang, Jianye Kang, Yuru Wu, Rong Bao, Niantong Li, Wei Wang, Ziyi Cheng, Xinfa Zhu, HangRui Hu, Ting He, Bing Zhao, Lin Qu, Hu Wei, Jin Xu

diffusion multimodal benchmarks

arXiv AI

Jun 26

Reconstruction Alignment Improves Unified Multimodal Models

arXiv:2509. 07295v4 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture.

By Ji Xie, Trevor Darrell, Luke Zettlemoyer, XuDong Wang

rag diffusion multimodal benchmarks safety

One email a morning, machine-written

Related stories

UOT-Gap: A Variational Principle for the Modality Gap in Vision-Language Models via Unbalanced Optimal Transport

Conan-embedding-v3: Fusing Modality-Specific Models for Omni-Modal Embedding

Hypergraph-Regularized Gramian Volumes for Multimodal Retrieval

WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report

OmniJudge or OmniBias? Diagnosing Multimodal Judges through Balanced, Decoupled Lenses

Reconstruction Alignment Improves Unified Multimodal Models