Masked language modelling (MLM) remains the dominant pretraining objective for transformer encoders on longitudinal electronic health records (EHRs), but reconstructing individual events can overemphasise noisy observations and fails to encourage learning higher‑level patient states. Joint‑Embedding Predictive Architectures (JEPAs), which predict representations in latent space, were tested on MIMIC‑IV data and, while training stably, consistently underperformed standard MLM. The study found that larger target sizes increase prediction error, yet reducing target size does not improve downstream performance, indicating that stable latent prediction alone is insufficient for competitive EHR representation learning and underscoring the difficulty of defining clinically meaningful latent targets for EHR.
arXiv:2607. 15447v1 Announce Type: new Abstract: Recent research in clinical machine learning, focusing on outcome predictions in intensive care unit (ICU), has shifted from bespoke supervised models to foundation models, utilising modern representation learning methods.
arXiv:2608. 20315v1 Announce Type: new Abstract: Predictive models over structured electronic health records (EHRs) remain central to machine learning for healthcare, but few have jointly emphasized quantitative laboratory information and interpretability with respect to input medical events.
Autoregressive (AR) models have proved effective for zero-shot prediction from longitudinal electronic health records, but impose strictly sequential generation on trajectories containing temporally c...
arXiv:2602. 00541v2 Announce Type: replace Abstract: Clinical events captured in Electronic Health Records (EHR) are irregularly sampled and may consist of a mixture of discrete events and numerical measurements, such as laboratory values or treatment dosages.
arXiv:2606. 28798v1 Announce Type: new Abstract: Objective: ICD codes are central to reimbursement, research, and population health surveillance, yet automated coding systems often struggle to integrate diagnostic signals from both clinical narratives and structured electronic health record (EHR) variables.
arXiv:2509. 24118v2 Announce Type: replace Abstract: Electronic health Records (EHRs) have become a cornerstone in modern-day healthcare.
arXiv:2608. 06430v1 Announce Type: new Abstract: Learning from Electronic Health Records (EHRs) has gained significant attention due to its potential to improve clinical prediction.
The paper evaluates block-based masked diffusion (BD) as an alternative to autoregressive (AR) models for zero‑shot prediction from longitudinal electronic health records. On the MIMIC‑IV dataset, BD shows a clear trade‑off between quality and efficiency: small‑block configurations retain most of AR’s predictive performance while enabling faster inference, with a representative B=8 setting achieving about 1.43× faster evaluation. Although BD’s generative rollout lags behind AR in most settings, frozen linear probes on BD and AR representations perform nearly identically across clinical prediction tasks, indicating that BD learns comparably informative clinical representations.
arXiv:2609.37587v1 Announce Type: cross Abstract: Longitudinal electronic health record (EHR) modeling requires integrating new visits with an expanding patient history. Yet the continual accumulatio...
PaReGTA is a large‑language‑model based framework that transforms longitudinal electronic health record events into visit‑level templated text with explicit temporal cues, learns visit embeddings through lightweight contrastive fine‑tuning, and aggregates them into fixed‑dimensional patient representations using hybrid temporal pooling. These representations can be fed into conventional machine‑learning models. In a cohort of 39,088 migraine patients from the All of Us Research Program, PaReGTA combined with LightGBM achieved the highest AUC, accuracy, and F1 scores on a held‑out test set compared to sparse, BERT‑based, and recurrent approaches.
arXiv:2606. 24102v1 Announce Type: cross Abstract: Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot directly represent events containing unseen concepts or new combinations of concepts and attributes such as numeric values.
arXiv:2607. 22264v1 Announce Type: new Abstract: Autoregressive foundation models trained on tokenized electronic health records (EHRs) can support zero-shot clinical prediction, yet most operate on structured event codes alone, and do not incorporate multiple modalities in a principled way.