Hugging Face Trending Papers

FedCoRe: Target-Adaptive Completion for Missing Modalities in Healthcare Federated Learning

FedCoRe is a federated learning framework that addresses missing modalities in healthcare by learning representation- or logit-space corrections instead of generating synthetic data. In a MIMIC-derived respiratory deterioration task, the method uses paired examples where a modality is present during training but may be absent at deployment, allowing only those clients to update the completion module. Experiments show that FedCoRe can recover roughly half of the performance lost when ECG or CXR data are hidden, but the framework should only be deployed when paired examples and validation evidence confirm the modality’s presence.

arXiv AI

Aug 20

FedCoRe: Target-Adaptive Completion for Missing Modalities in Healthcare Federated Learning

FedCoRe is a federated learning framework that addresses missing modalities in healthcare by learning representation- or logit-space corrections instead of generating synthetic data. In a MIMIC-derived respiratory deterioration task, the method uses paired examples where a modality is present or absent to train a completion module, achieving partial recovery of performance lost when modalities like ECG or CXR are hidden. The approach emphasizes validation-gated deployment, ensuring that completion is only applied when paired examples and validation evidence support the presence of the missing modality.

By Holger R. Roth, Ziyue Xu, Peter Cnudde

multimodal

arXiv AI

Aug 11

Multimodal Federated Learning under Dual-Axis Modality Missingness

arXiv:2608. 09240v1 Announce Type: cross Abstract: Multimodal federated learning (FL) supports collaborative modeling in privacy-sensitive health-sensing and medical settings, but realistic deployments often exhibit dual-axis modality missingness: clients have different modality sets, and individual samples may contain only subsets of the modalities available locally.

By Adiba Orzikulova, Jaehyun Kwak, Jaemin Shin, Yunqi Guo, Xiaomin Ouyang, Guoliang Xing, Steven Euijong Whang, Sung-Ju Lee

multimodal safety

arXiv AI

Jul 9

ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities

arXiv:2607. 06633v1 Announce Type: cross Abstract: In this paper, we address the problem of multimodal federated learning with missing modality.

By Aavash Chhetri, Bibek Niroula, Eduard Vazquez, Yash Raj Shrestha, Prashnna Gyawali, Loris Bazzani, Binod Bhattarai

multimodal benchmarks

arXiv AI

2d ago

Multimodal Prompt Learning with Irregular EHRs for Robust Monitoring of Critical Care Patients

arXiv:2608.21941v1 Announce Type: new Abstract: Accurate assessment of patients in intensive care units (ICUs) is essential for timely clinical intervention and improved patient outcomes. Multimodal...

By Yixin Yang, Yueyang Sun, Weichen Liu, Xianbing Zhao, Sicen Liu

multimodal benchmarks

arXiv Machine Learning

Jul 14

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

arXiv:2607. 09982v1 Announce Type: new Abstract: Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance.

By Nikkie Hooman, Zhongjie Wu, Eric C. Larson, Mehak Gupta

multimodal safety

arXiv Machine Learning

Aug 17

MedMix: Specialization-Consistent Federated Sparse MoEs under Modality Heterogeneity

arXiv:2608. 13911v1 Announce Type: new Abstract: Federated multimodal medical AI faces modality heterogeneity at both the client and sample levels: clients may systematically lack access to specific modality types, while individual records within the same client may contain different partial modality subsets.

By Adiba Orzikulova, Dong Min Kim, Jaehong Yoon, Sung-Ju Lee

multimodal safety

arXiv Machine Learning

Jul 27

Autoregressive EHR Foundation Models with Multimodal Inputs

arXiv:2607. 22264v1 Announce Type: new Abstract: Autoregressive foundation models trained on tokenized electronic health records (EHRs) can support zero-shot clinical prediction, yet most operate on structured event codes alone, and do not incorporate multiple modalities in a principled way.

By Yuxuan Liu, Joshua Placidi, Jinpei Han, Alfred John Balston, Marek Rei, A. Aldo Faisal

multimodal safety

arXiv AI

Jun 11

Latent World Recovery for Multimodal Learning with Missing Modalities

arXiv:2606. 12362v1 Announce Type: cross Abstract: We study multimodal learning under missing modalities, with particular motivation from bioscience applications in which heterogeneous modalities are often only partially available when decisions need to be made.

By Hui Wang, Tianyu Ren, Joseph Butler, Christopher Baker, Karen Rafferty, Simon McDade

rag diffusion multimodal benchmarks safety

arXiv Machine Learning

Jul 14

ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning

arXiv:2607. 09884v1 Announce Type: cross Abstract: Multimodal medical models often degrade when inputs are missing, a common scenario in real-world clinical workflows.

By Nusrat Binta Nizam, Fengbei Liu, Sunwoo Kwak, Minh Nguyen, Ruining Deng, Mert R. Sabuncu

multimodal

arXiv Machine Learning

Jun 5

PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Data

arXiv:2606. 06328v1 Announce Type: new Abstract: In healthcare, multimodal time series tasks often operate on incomplete observations in practice, for example when ECG segments are lost because electrodes detach or an entire respiratory channel is unavailable during overnight monitoring.

By Ziwen Kan, Wugeng Zheng, Tianlong Chen, Song Wang

rag diffusion multimodal benchmarks

arXiv Machine Learning

Jul 28

FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities

arXiv:2607. 23245v1 Announce Type: cross Abstract: Multimodal Federated Learning is often challenged by arbitrary modality missingness and Non-IID data distributions, which lead to severe representation drift and hinder effective collaboration across clients.

By Haochen Liang, Jie Zhang, Hideya Ochiai

multimodal safety

arXiv Machine Learning

Aug 14

Represent, Then Generate: Multimodal-Conditioned Time-Series Generation under Irregular Missingness

arXiv:2608. 12592v1 Announce Type: new Abstract: Continuous physiological time series underpin modern clinical monitoring, yet many of the most informative signals are invasive, expensive, or simply unavailable for a given patient.

By Haochen Zhang, Jiaheng Guo, Yu-Chao Huang, Nicholas Knoz, Tianlong Chen

multimodal benchmarks

Related stories