Are Joint-Embedding Predictive Objectives Effective for Longitudinal EHRs?
Read the original on arXiv Machine Learning →Masked language modelling (MLM) remains the dominant pretraining objective for transformer encoders on longitudinal electronic health records (EHRs), but reconstructing individual events can overemphasise noisy observations and fails to encourage learning higher‑level patient states. Joint‑Embedding Predictive Architectures (JEPAs), which predict representations in latent space, were tested on MIMIC‑IV data and, while training stably, consistently underperformed standard MLM. The study found that larger target sizes increase prediction error, yet reducing target size does not improve downstream performance, indicating that stable latent prediction alone is insufficient for competitive EHR representation learning and underscoring the difficulty of defining clinically meaningful latent targets for EHR.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.