UMER is a Unified Multimodal Embedding and Ranking framework that combines contrastive embeddings with Pair‑Aware Discriminative Reasoning to improve universal multimodal retrieval. It replaces item‑wise reflection with pair‑wise comparison of query–candidate pairs, enabling explicit identification of matching and discrepancy evidence. A mutual distillation strategy transfers reliable pairwise preferences between the embedding and ranking components, and UMER achieves state‑of‑the‑art performance on the MMEB‑V2 benchmark while supporting budget‑adjustable inference.
By Libiao Chen, Xiyang Liu, Yanheng Wei, Tao Wang, Zhenyu Tang
arXiv:2609.05518v1 Announce Type: cross
Abstract: Despite the strong capabilities of multimodal large language models (MLLMs), their parametric knowledge remains incomplete and difficult to update, m...
By Jiacheng Cai, Zijin Hong, Zheng Yuan, Huachi Zhou, Qinggang Zhang, Xiao Huang
arXiv:2604.22280v4 Announce Type: replace
Abstract: Multimodal Large Language Models (MLLMs) have emerged as a promising foundation for universal multimodal embeddings. Recent studies have shown that...
By Peixi Wu, Ke Mei, Feipeng Ma, Bosong Chai, Zhibin Lan, Chenxi Zhao, Shannan Yan, Jie Chen, Zhangchi Hu, Yansong Peng, Bo Lin, Junjie Zhou, Dacheng Yin, Tianyi Wang, Fengyun Rao, Jing Lyu, Hebei Li, Xiaoyan Sun
Lingshu is a medical‑specialized multimodal large language model that addresses key limitations of existing medical MLLMs, such as narrow knowledge coverage, hallucinations, and weak reasoning. The authors curate a comprehensive dataset combining medical imaging, texts, and general‑domain data, then train Lingshu in multiple stages to embed medical expertise and improve task performance. They also introduce MedEvalKit, a unified evaluation framework, and demonstrate that Lingshu outperforms current open‑source multimodal models on multimodal QA, text‑based QA, and medical report generation.
By Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, Yu Sun, Junao Shen, Chaojun Wang, Jie Tan, Deli Zhao, Tingyang Xu, Hao Zhang, Yu Rong
arXiv:2505.13520v2 Announce Type: replace-cross
Abstract: Textbook question answering (TQA) is a complex task, requiring the interpretation of complex multimodal context. Although recent advances hav...
By Hessa Alawwad, Usman Naseem, Areej Alhothali, Ali Alkhathlan, Amani Jamal
arXiv:2607. 09623v1 Announce Type: cross Abstract: We present our submission to the QANTA 2026 shared challenge at the ICML 2026 Workshop on Efficient Multimodal Question Answering (EMM-QA).
By Nirjhar Das, Md. Al-Mamun Provath