Masked language modelling (MLM) remains the dominant pretraining objective for transformer encoders on longitudinal electronic health records (EHRs), but reconstructing individual events can overemphasise noisy observations and fails to encourage learning higher‑level patient states. Joint‑Embedding Predictive Architectures (JEPAs), which predict representations in latent space, were tested on MIMIC‑IV data and, while training stably, consistently underperformed standard MLM. The study found that larger target sizes increase prediction error, yet reducing target size does not improve downstream performance, indicating that stable latent prediction alone is insufficient for competitive EHR representation learning and underscoring the difficulty of defining clinically meaningful latent targets for EHR.
By Ashvin Gupta, Brendan Delaney, Alessandra Russo
The paper introduces Latent-Attention Masked Autoencoder (LAMAE), a multimodal masked autoencoder that learns patient-level representations by exchanging information directly in the latent space through a shared latent-attention module. LAMAE integrates complementary modalities—electrocardiogram, echocardiography, and chest X-rays—within a study-view-entity hierarchy, enabling aggregation of variable observations and handling missing modalities. Pretraining on over 500,000 MIMIC-IV hospital stays, LAMAE outperforms modality-specific pretraining and strong contrastive and vision-language baselines on multimodal hospital-stay tasks such as in-hospital mortality, ICD-10 and DRG coding, and length of stay, while remaining competitive on unimodal tasks.
By Andrea Agostini, Simon B\"ohi, Moritz Vandenhirtz, Samuel Ruiperez-Campillo, Max Kr\"ahenmann, Silke M\"uhlstedt, Irene Cannistraci, Ece \"Ozkan Elsen, Julia E. Vogt, Thomas M. Sutter
Fed-ReMasker is a federated learning approach that adapts the ReMasker masked autoencoder for tabular data imputation, specifically addressing feature-level missingness where entire features are absent at some centers. The method enables centers to impute unobserved features by leveraging knowledge from collaborating institutions. In benchmark tests on synthetic and real-world datasets, Fed-ReMasker achieves the lowest imputation error in the majority of scenarios and remains robust to client heterogeneity, closely matching the performance of a centralized model.
By Ioannis Papathanail, Rooholla Poursoleymani, Lubnaa Abdur Rahman, Stavroula Georgia Mougiakakou
Autoregressive (AR) models have proved effective for zero-shot prediction from longitudinal electronic health records, but impose strictly sequential generation on trajectories containing temporally c...
The paper evaluates block-based masked diffusion (BD) as an alternative to autoregressive (AR) models for zero‑shot prediction from longitudinal electronic health records. On the MIMIC‑IV dataset, BD shows a clear trade‑off between quality and efficiency: small‑block configurations retain most of AR’s predictive performance while enabling faster inference, with a representative B=8 setting achieving about 1.43× faster evaluation. Although BD’s generative rollout lags behind AR in most settings, frozen linear probes on BD and AR representations perform nearly identically across clinical prediction tasks, indicating that BD learns comparably informative clinical representations.
By Ashvin Gupta, Joshua Placidi, Brendan Delaney, Alessandra Russo
arXiv:2607. 22264v1 Announce Type: new Abstract: Autoregressive foundation models trained on tokenized electronic health records (EHRs) can support zero-shot clinical prediction, yet most operate on structured event codes alone, and do not incorporate multiple modalities in a principled way.
By Yuxuan Liu, Joshua Placidi, Jinpei Han, Alfred John Balston, Marek Rei, A. Aldo Faisal
arXiv:2606. 15284v1 Announce Type: cross Abstract: Photoplethysmography (PPG) plays a central role in wearable health monitoring and clinical decision support.
By Chenyang He, Xinyi Shao, Shun Huang, Bosong Huang, Daoqiang Zhang, Ming Jing, Cheng Ding
arXiv:2608. 10969v1 Announce Type: new Abstract: Healthcare data, such as Intensive Care Unit (ICU) records, comprise heterogeneous multivariate time series sampled at irregular intervals with pervasive missingness.
By Ruirui Wang, Yanke Li, Manuel G\"unther, Diego Paez-Granados
arXiv:2606. 06328v1 Announce Type: new Abstract: In healthcare, multimodal time series tasks often operate on incomplete observations in practice, for example when ECG segments are lost because electrodes detach or an entire respiratory channel is unavailable during overnight monitoring.
By Ziwen Kan, Wugeng Zheng, Tianlong Chen, Song Wang
arXiv:2607. 16681v1 Announce Type: new Abstract: Early sepsis prediction from electronic health records is challenged by irregular sampling, high missingness, and class imbalance.
By Umair bin Mansoor, Munaf Rashid, Roomi Naqvi
arXiv:2610.04946v2 Announce Type: replace
Abstract: Cross-system transfer of longitudinal Electronic Health Record (EHR) representations is challenging because clinical coding, patient populations, a...
By Qingyang Zhang
arXiv:2604.04614v3 Announce Type: replace
Abstract: Deep learning-based modeling of multimodal Electronic Health Records (EHRs) has become an important approach for clinical diagnosis and risk predic...
By Bohao Li, Tao Zou, Junchen Ye, Yan Gong, Bowen Du