arXiv:2602. 15159v2 Announce Type: replace Abstract: Electronic health records (EHR) arrive masked.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
Masked language modelling (MLM) remains the dominant pretraining objective for transformer encoders on longitudinal electronic health records (EHRs), but reconstructing individual events can overemphasise noisy observations and fails to encourage learning higher‑level patient states. Joint‑Embedding Predictive Architectures (JEPAs), which predict representations in latent space, were tested on MIMIC‑IV data and, while training stably, consistently underperformed standard MLM. The study found that larger target sizes increase prediction error, yet reducing target size does not improve downstream performance, indicating that stable latent prediction alone is insufficient for competitive EHR representation learning and underscoring the difficulty of defining clinically meaningful latent targets for EHR.
The paper introduces Latent-Attention Masked Autoencoder (LAMAE), a multimodal masked autoencoder that learns patient-level representations by exchanging information directly in the latent space through a shared latent-attention module. LAMAE integrates complementary modalities—electrocardiogram, echocardiography, and chest X-rays—within a study-view-entity hierarchy, enabling aggregation of variable observations and handling missing modalities. Pretraining on over 500,000 MIMIC-IV hospital stays, LAMAE outperforms modality-specific pretraining and strong contrastive and vision-language baselines on multimodal hospital-stay tasks such as in-hospital mortality, ICD-10 and DRG coding, and length of stay, while remaining competitive on unimodal tasks.
Fed-ReMasker is a federated learning approach that adapts the ReMasker masked autoencoder for tabular data imputation, specifically addressing feature-level missingness where entire features are absent at some centers. The method enables centers to impute unobserved features by leveraging knowledge from collaborating institutions. In benchmark tests on synthetic and real-world datasets, Fed-ReMasker achieves the lowest imputation error in the majority of scenarios and remains robust to client heterogeneity, closely matching the performance of a centralized model.
Autoregressive (AR) models have proved effective for zero-shot prediction from longitudinal electronic health records, but impose strictly sequential generation on trajectories containing temporally c...
The paper evaluates block-based masked diffusion (BD) as an alternative to autoregressive (AR) models for zero‑shot prediction from longitudinal electronic health records. On the MIMIC‑IV dataset, BD shows a clear trade‑off between quality and efficiency: small‑block configurations retain most of AR’s predictive performance while enabling faster inference, with a representative B=8 setting achieving about 1.43× faster evaluation. Although BD’s generative rollout lags behind AR in most settings, frozen linear probes on BD and AR representations perform nearly identically across clinical prediction tasks, indicating that BD learns comparably informative clinical representations.
arXiv:2607. 22264v1 Announce Type: new Abstract: Autoregressive foundation models trained on tokenized electronic health records (EHRs) can support zero-shot clinical prediction, yet most operate on structured event codes alone, and do not incorporate multiple modalities in a principled way.