arXiv Machine Learning

DeliMF: Gated Cross-Attention Fusion of Structured Time Series and Clinical Text for Early Intensive Care Delirium Prediction

arXiv Machine Learning

Jul 27

Autoregressive EHR Foundation Models with Multimodal Inputs

arXiv:2607. 22264v1 Announce Type: new Abstract: Autoregressive foundation models trained on tokenized electronic health records (EHRs) can support zero-shot clinical prediction, yet most operate on structured event codes alone, and do not incorporate multiple modalities in a principled way.

By Yuxuan Liu, Joshua Placidi, Jinpei Han, Alfred John Balston, Marek Rei, A. Aldo Faisal

multimodal safety

arXiv AI

Aug 17

Removing Temporal Note Redundancy Improves Multimodal Reinforcement Learning for Medicine

arXiv:2608. 14157v1 Announce Type: new Abstract: Mechanical ventilation is a critical life-support intervention, requiring dynamic adjustments to ventilator settings as a patient's condition evolves.

By Chenran Weng, Joo Seung Lee, Malini Mahendra, Anil Aswani

rag reinforcement-learning multimodal

arXiv AI

Sep 1

MedTVL: Harnessing Vision and Language for Medical Time Series Classification

arXiv:2608.28605v1 Announce Type: new Abstract: Recent advancements in multimodal learning for medical time series (MedTS) classification highlight the benefits of integrating complementary modalitie...

By Jiexia Ye, Jia Li, Fugee Tsung

llms multimodal

arXiv AI

Oct 5

RASPER: Reward-Aligned Summarization of Clinical Notes for EHR Outcome Prediction

RASPER is a reward‑aligned summarizer that tailors the extraction of information from unstructured discharge notes to improve downstream clinical predictions. It uses a tunable LLM summarizer trained with reinforcement learning, where the reward comes from the loss of a downstream predictor, and incorporates patient‑specific context via a longitudinal encoder that soft‑prompts the summarizer with structured codes. The approach consistently outperforms strong baselines on readmission prediction and medication recommendation tasks in the MIMIC‑III and MIMIC‑IV datasets.

By Arya Hadizadeh Moghaddam, Mohsen Nayebi Kerdabadi, Chen Chen, Dongjie Wang, Zijun Yao

llms nlp reinforcement-learning multimodal

arXiv Machine Learning

Aug 4

EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records

arXiv:2506. 04831v3 Announce Type: replace Abstract: Forecasting how a patient's condition is likely to evolve, including possible deterioration, recovery, treatment needs, and care transitions, could support more proactive and personalized care, but requires modeling heterogeneous and longitudinal electronic health record (EHR) data.

By Chantal Pellegrini, Ege \"Ozsoy, David Bani-Harouni, Matthias Keicher, Nassir Navab

nlp multimodal

arXiv AI

Jun 30

Primary ICD Category Prediction using LLM-based Probing

arXiv:2606. 28798v1 Announce Type: new Abstract: Objective: ICD codes are central to reimbursement, research, and population health surveillance, yet automated coding systems often struggle to integrate diagnostic signals from both clinical narratives and structured electronic health record (EHR) variables.

By Chengyuan Liu, Xinyue Zhang, Yao Li, Guanting Chen

llms rag multimodal

arXiv Machine Learning

Jun 24

PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models

arXiv:2606. 24102v1 Announce Type: cross Abstract: Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot directly represent events containing unseen concepts or new combinations of concepts and attributes such as numeric values.

By Lin Lawrence Guo, Adam Paul Yan, Emily Vettese, Lillian Sung

arXiv AI

Jul 20

Large Language Models as Unified Multimodal Learners for Clinical Prediction

arXiv:2607. 15380v1 Announce Type: cross Abstract: Electronic health records combine free-text clinical narratives with structured measurements such as vital signs, laboratory values, and comorbidities.

By Ajay Madhavan Ravichandran, Bilgin Osmandoja, Klemens Budde, Klaus Netter, Tobias Strapatsas, Aljoscha Burchardt, Sebastian M\"oller, Roland Roller

llms fine-tuning multimodal

arXiv Machine Learning

Jul 20

LLM4EHR: Aligning Clinical Time Series with Medical Event Sequences via Large Language Models

arXiv:2607. 15447v1 Announce Type: new Abstract: Recent research in clinical machine learning, focusing on outcome predictions in intensive care unit (ICU), has shifted from bespoke supervised models to foundation models, utilising modern representation learning methods.

By Jingteng Li, Alexander Capstick, Louise Rigny, Iona Biggart, Neil J Sebire, Payam Barnaghi

llms rag benchmarks

arXiv AI

Aug 25

Multimodal Prompt Learning with Irregular EHRs for Robust Monitoring of Critical Care Patients

arXiv:2608.21941v1 Announce Type: new Abstract: Accurate assessment of patients in intensive care units (ICUs) is essential for timely clinical intervention and improved patient outcomes. Multimodal...

By Yixin Yang, Yueyang Sun, Weichen Liu, Xianbing Zhao, Sicen Liu

multimodal benchmarks

arXiv Machine Learning

Oct 2

Learning to Select Source-Traceable Evidence for Language-Model Prediction from Irregular Clinical Time Series

arXiv:2605.20292v2 Announce Type: replace Abstract: Numerical time-series models effectively process irregular electronic health record (EHR) trajectories, but do not expose which temporal patterns s...

By Kwanhyung Lee, Juhwan Choi, Jongheon Kim, Joohyung Lee, Hyeongwon Jang, Jeonguk Lee, Jisoo Jung, Eunho Yang

llms benchmarks

arXiv AI

Jun 16

Fusion is not one-size-fits-all: Cross-Modal Representation Alignment for Time-to-Event Modeling

arXiv:2606. 15038v1 Announce Type: new Abstract: Accurate time-to-event (TTE) prediction from multimodal clinical data remains challenging due to modality imbalance and distribution shift.

By Zhemin Zhang, Weijie Chen, David Le, Amara Tariq, Alex Wallace, Matthew Stib, Juan Maria Farina, Chadi Ayoub, Reza Arsanjani, Imon Banerjee

diffusion multimodal safety

Related stories