arXiv Computer Vision October 9, 2026 By Tuan Truong, Melanie Dohmen, Sara Lorio, Matthias Lenga

Revisiting Integration of Image and Metadata for DICOM Series Classification: Cross-Attention and Dictionary Learning

Read the original on arXiv Computer Vision →

The paper presents an end‑to‑end multimodal framework for classifying DICOM image series that jointly processes image content and acquisition metadata. It uses modality‑aware encoders, a bi‑directional cross‑modal attention mechanism, and a sparse, missingness‑aware dictionary‑based encoder for metadata, avoiding imputation. The method handles variable series length with a 2.5D visual encoder and demonstrates superior performance on the Duke Liver MRI dataset and a large multi‑institutional cohort, outperforming image‑only, metadata‑only, and other multimodal baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Jun 16

Unified Multimodal Model for Brain MRI Imputation and Understanding

arXiv:2606. 16484v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) hold great potential for medicine, as they inherit knowledge from LLM and allow multiple data modalities to be integrated, analysed and interpreted in natural language.

By Zhiyun Song, Che Liu, Tian Xia, Avinash Kori, Wenjia Bai

llms rag multimodal safety

arXiv Machine Learning

Jul 9

MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models

arXiv:2607. 07673v1 Announce Type: cross Abstract: Medicine is inherently multimodal, requiring clinicians to synthesize information across diverse data streams.

By Hyunjae Kim, Dain Kim, Pan Xiao, Serina S. Applebaum, Younjoon Chung, Xuguang Ai, Yu Yin, Roy Jiang, Yuexi Du, Yawen Wei, Yiming Kong, Tuo Guo, Zhiyuan Cao, Mengmeng Du, Yuelei Fu, Yan Hu, Rui Shi, Gui Yang, Kevin W. Jin, Yuntian Liu, Yuxuan Tian, Jonathan Marquez, Zhen Chen, Sheng Zhang, Hoifung Poon, Hua Xu, Jaewoo Kang, Qingyu Chen

llms multimodal benchmarks safety

arXiv Computer Vision

Sep 22

Metadata Supervised Imaging Representations for Modelling and Controlling Acquisition Variability

arXiv:2607.11295v2 Announce Type: replace Abstract: Biomedical imaging data exhibit substantial acquisition variability, where identical biological structures can appear markedly different due to dif...

By Mehmet Yigit Avci, Pedro Borges, Virginia Fernandez, Natalia Glazman, Paul Wright, Mehmet Yigitsoy, Sebastien Ourselin, Jorge Cardoso

safety

Hugging Face Trending Papers

Jul 21

Advancing Multimodal Fusion on Heterogeneous Medical Data with Hybrid Geometry Attention

Multimodal fusion learning (MFL) has shown great potential in the medical domain, where we are faced with disparate data modalities such as imaging, clinical records, and omics. However, existing MFL strategies face several major challenges.

multimodal

arXiv Computer Vision

Aug 25

Expert-level vision-language foundation model for real-world radiology and comprehensive evaluation

arXiv:2409.16183v2 Announce Type: replace Abstract: Radiology is a vital and complex component of modern clinical workflow and covers many tasks. Recently, vision-language (VL) foundation models in m...

By Xiaohong Liu, Guoxing Yang, Yulin Luo, Jiaji Mao, Xiang Zhang, Haibo Wang, Zhiyang He, Ming Gao, Shanghang Zhang, Jun Shen, Guangyu Wang

multimodal benchmarks

arXiv Machine Learning

Jul 17

Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging

arXiv:2607. 14995v1 Announce Type: new Abstract: Multimodal Contrastive Learning (CL) has shown significant performance in aligning representations across various data modalities and improving downstream tasks, especially in healthcare.

By Sara Ketabi, Matthias W. Wagner, Cynthia Hawkins, Uri Tabori, Birgit Betina Ertl-Wagner, Farzad Khalvati