arXiv:2602. 18400v3 Announce Type: replace-cross Abstract: Missing data problems, such as missing modalities in multi-modal brain MRI and missing slices in cardiac MRI, pose significant challenges in clinical practice.
The paper presents a newly curated, multi-center, multi-modal, and longitudinal lung cancer dataset comprising 1,365 patients with whole-slide images, CT scans, PET scans, structured clinical data, transcriptomics, and follow-up information. The dataset features substantial, non-uniform missingness across modalities, making it ideal for evaluating robust multi-modal fusion strategies. Benchmarks on 12‑month overall survival, disease‑specific survival, and longitudinal hazard prediction demonstrate that integrating complementary modalities consistently outperforms uni-modal approaches, even under severe missing data.
arXiv:2607. 06633v1 Announce Type: cross Abstract: In this paper, we address the problem of multimodal federated learning with missing modality.
arXiv:2606. 16484v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) hold great potential for medicine, as they inherit knowledge from LLM and allow multiple data modalities to be integrated, analysed and interpreted in natural language.
The paper presents an end‑to‑end multimodal framework for classifying DICOM image series that jointly processes image content and acquisition metadata. It uses modality‑aware encoders, a bi‑directional cross‑modal attention mechanism, and a sparse, missingness‑aware dictionary‑based encoder for metadata, avoiding imputation. The method handles variable series length with a 2.5D visual encoder and demonstrates superior performance on the Duke Liver MRI dataset and a large multi‑institutional cohort, outperforming image‑only, metadata‑only, and other multimodal baselines.
Multimodal fusion learning (MFL) has shown great potential in the medical domain, where we are faced with disparate data modalities such as imaging, clinical records, and omics. However, existing MFL strategies face several major challenges.
MMAP is a Multimodal Missing‑Aware Alignment Pretraining method designed to learn image‑tabular representations from incomplete data. It uses a sigmoid contrastive learning image encoder with generative reconstruction, a tabular encoder based on a foundation model, and a missing token generator to handle missing modalities. The approach is evaluated on longitudinal Alzheimer’s tasks—predicting disease stage conversion and amyloid status—and outperforms both multimodal and unimodal baselines.
The paper introduces the FAPT-M Dataset, a pathology‑confirmed multimodal collection of 910 breast patients that includes ultrasound images, structured clinical attributes, and diagnostic descriptions. It proposes a clinically guided multimodal framework that fuses DenseNet visual encoding, CLIP‑inspired text encoding, and lightweight clinical encoding, enhanced by clinical‑conditioned adaptive modulation, cross‑modal Transformer fusion, and dual‑path representation learning. The method achieves 77.64% accuracy, 73.38% F1‑score, and 89.74% AUC, outperforming CNN, Transformer, and vision‑language baselines, and demonstrates the value of three‑modality fusion through ablation and class‑balanced studies.
Color Fundus Photography (CFP) is a primary non-invasive imaging modality for large-scale screening of ophthalmic and systemic diseases. Existing surveys mainly summarize task-specific algorithms, datasets, or preprocessing techniques independently, lacking a unified perspective on their co-evolution with modern artificial intelligence.
The paper presents a method for generating cardiac magnetic resonance (CMR) images conditioned on patient metadata using a pretrained latent diffusion model. By encoding structured clinical data and slice position as textual prompts and applying Metadata‑Free Classifier‑Free Guidance, Contrastive Batching, and Inverse‑Frequency Sampling, the authors improve the fidelity of synthetic images, achieving a 57% reduction in Fréchet Inception Distance compared to a baseline without these strategies. Evaluation on 59,058 UK Biobank CMR scans shows better distributional realism and subgroup alignment, though disease‑specific conditioning remains challenging.
The paper introduces a multimodal latent diffusion model that jointly synthesizes volumetric MRI scans and tabular clinical data within a shared latent space using cross‑attention. By fusing the modalities through a variational autoencoder before diffusion, the model reconstructs MRI and tabular data with separate decoders, producing anatomically plausible images that align with synthesized clinical attributes. Evaluation on the German National Cohort dataset shows high‑fidelity image generation and superior tabular performance compared to CTGAN, marking the first demonstration of joint MRI‑tabular modeling in a single diffusion framework.
arXiv:2606. 11107v1 Announce Type: cross Abstract: Clinicians diagnose brain tumors by synthesizing patient symptoms, medical history, and quantitative imaging data from modalities such as MRI and CT scans into a unified clinical judgement.