arXiv Machine Learning September 7, 2026 By Yuqing Yang, Alexander Schmatz, Zhaozhao Ma, Changkyu Choi, Robert Jenssen, Shujian Yu

SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis

Read the original on arXiv Machine Learning →

The paper introduces SMILE, a self‑explainable multimodal information bottleneck framework for medical diagnosis. It jointly optimizes predictive accuracy and modality‑specific explainability by selecting the most informative elements within each data modality. Experiments on diverse medical datasets show strong diagnostic performance, including a 9.1‑percentage‑point accuracy gain on the iCTCF dataset, and provide transparent, modality‑aware explanations that enhance both explainability and generalization.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 14

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

arXiv:2607. 09982v1 Announce Type: new Abstract: Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance.

By Nikkie Hooman, Zhongjie Wu, Eric C. Larson, Mehak Gupta

multimodal safety

arXiv AI

Aug 21

MKG-CARE: Case-Aware Reasoning with Multimodal Knowledge Graphs for Explainable Medical Image Diagnosis

arXiv:2605. 22547v3 Announce Type: replace-cross Abstract: Medical image diagnosis has achieved significant progress with deep learning, yet existing methods often rely on isolated visual evidence and lack the ability to effectively leverage similar cases and external knowledge.

By Yiming Xu, Yixuan Liu, Yuhang Zhang, Ling Zheng, Yihan Wang, Qi Song

multimodal benchmarks safety

arXiv Machine Learning

Jul 13

Contrastive Learning on Multimodal Analysis of Electronic Health Records

arXiv:2403. 14926v3 Announce Type: replace-cross Abstract: Electronic health record (EHR) systems capture a wealth of multimodal clinical data, encompassing both structured clinical codes and unstructured clinical notes.

By Tianxi Cai, Feiqing Huang, Ryumei Nakada, Linjun Zhang, Doudou Zhou

rag multimodal safety

arXiv AI

Jul 29

Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases

arXiv:2607. 25933v1 Announce Type: cross Abstract: Clinical diagnostic evaluation should not only assess whether models can provide correct diagnoses, but also reflect the realities of clinical practice, including progressive disclosure of multimodal information, dynamic updating of diagnostic hypotheses, and continuous refinement of clinical reasoning.

By Rui Yang, Weihao Xuan, Yi Lin, Zhuhan Bao, Jonathan Chong Kai Liew, Matthew Yu Heng Wong, Nicol\'as Lescano, Nikita R. Paripati, Emily Ling-Lin Pai, Jiarui Liu, Heli Qi, Heng-Jui Chang, Benny Kai Guo Loo, Huitao Li, Kunyu Yu, Yufan Wang, Chuan Hong, Shijian Lu, Douglas Teodoro, Naoto Yokoya, Ross Koppel, Mona Diab, Hua Xu, David W. Bates, Nan Liu, Yifan Peng

llms multimodal benchmarks safety

arXiv Machine Learning

Aug 5

PatTree: a novel approach for automated creation of multimodal, graph-based patient representations for medical classification tasks

arXiv:2608. 02692v1 Announce Type: new Abstract: Access to holistic, multimodal data improves the performance of Artificial Intelligence (AI) in medical classification tasks compared to utilizing single modalities or data sources.

By Julia Gehrmann, Lars Quakulinski, Hamza Naseem, Oya Beyan

multimodal benchmarks

arXiv Machine Learning

Jul 7

OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction

arXiv:2604. 16878v2 Announce Type: replace Abstract: Early prediction of severe clinical deterioration and remaining length of stay can enable timely intervention and better resource allocation in high-acuity settings such as the ICU.

By Zhongyuan Liang, Junhyung Jo, Hyang-Jung Lee, Sang Kyu Kim, Irene Y. Chen

fine-tuning efficiency multimodal benchmarks

One email a morning, machine-written

Related stories

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

MKG-CARE: Case-Aware Reasoning with Multimodal Knowledge Graphs for Explainable Medical Image Diagnosis

Contrastive Learning on Multimodal Analysis of Electronic Health Records

Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases

PatTree: a novel approach for automated creation of multimodal, graph-based patient representations for medical classification tasks

OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction