The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper evaluates block-based masked diffusion (BD) as an alternative to autoregressive (AR) models for zero‑shot prediction from longitudinal electronic health records. On the MIMIC‑IV dataset, BD shows a clear trade‑off between quality and efficiency: small‑block configurations retain most of AR’s predictive performance while enabling faster inference, with a representative B=8 setting achieving about 1.43× faster evaluation. Although BD’s generative rollout lags behind AR in most settings, frozen linear probes on BD and AR representations perform nearly identically across clinical prediction tasks, indicating that BD learns comparably informative clinical representations.
Masked language modelling (MLM) remains the dominant pretraining objective for transformer encoders on longitudinal electronic health records (EHRs), but reconstructing individual events can overemphasise noisy observations and fails to encourage learning higher‑level patient states. Joint‑Embedding Predictive Architectures (JEPAs), which predict representations in latent space, were tested on MIMIC‑IV data and, while training stably, consistently underperformed standard MLM. The study found that larger target sizes increase prediction error, yet reducing target size does not improve downstream performance, indicating that stable latent prediction alone is insufficient for competitive EHR representation learning and underscoring the difficulty of defining clinically meaningful latent targets for EHR.
arXiv:2602. 15159v2 Announce Type: replace Abstract: Electronic health records (EHR) arrive masked.
arXiv:2607. 22264v1 Announce Type: new Abstract: Autoregressive foundation models trained on tokenized electronic health records (EHRs) can support zero-shot clinical prediction, yet most operate on structured event codes alone, and do not incorporate multiple modalities in a principled way.
arXiv:2506. 04831v3 Announce Type: replace Abstract: Forecasting how a patient's condition is likely to evolve, including possible deterioration, recovery, treatment needs, and care transitions, could support more proactive and personalized care, but requires modeling heterogeneous and longitudinal electronic health record (EHR) data.
arXiv:2608. 20315v1 Announce Type: new Abstract: Predictive models over structured electronic health records (EHRs) remain central to machine learning for healthcare, but few have jointly emphasized quantitative laboratory information and interpretability with respect to input medical events.