The paper introduces SMILE, a self‑explainable multimodal information bottleneck framework for medical diagnosis. It jointly optimizes predictive accuracy and modality‑specific explainability by selecting the most informative elements within each data modality. Experiments on diverse medical datasets show strong diagnostic performance, including a 9.1‑percentage‑point accuracy gain on the iCTCF dataset, and provide transparent, modality‑aware explanations that enhance both explainability and generalization.
By Yuqing Yang, Alexander Schmatz, Zhaozhao Ma, Changkyu Choi, Robert Jenssen, Shujian Yu
arXiv:2606.28798v2 Announce Type: replace
Abstract: The main objective of this paper is to propose a general framework for prediction based on different sources of multimodal data in the healthcare d...
By Chengyuan Liu, Xinyue Zhang, Yao Li, Guanting Chen
arXiv:2607. 09982v1 Announce Type: new Abstract: Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance.
By Nikkie Hooman, Zhongjie Wu, Eric C. Larson, Mehak Gupta
arXiv:2505. 06945v5 Announce Type: replace Abstract: Multimodal data modeling has emerged as a powerful approach in clinical research, enabling the integration of diverse data types such as imaging, genomics, wearable sensors, and electronic health records.
By Maryam Farhadizadeh, Maria Weymann, Michael Bla{\ss}, Johann Kraus, Christopher Gundler, Sebastian Walter, Noah Hempen, Hannah Bast, Harald Binder, Nadine Binder
arXiv:2607. 21496v1 Announce Type: cross Abstract: Cognitive impairment (CI) is a growing public health concern.
By Yingchao Huang, Xin Wang, Yuhan Su, Shanshan Yao
arXiv:2606. 16484v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) hold great potential for medicine, as they inherit knowledge from LLM and allow multiple data modalities to be integrated, analysed and interpreted in natural language.
By Zhiyun Song, Che Liu, Tian Xia, Avinash Kori, Wenjia Bai