← Back to all news
arXiv Machine Learning By Inhyeok Lee, Luke Solo, Michael C. Burkhart, Bashar Ramadan, Sahil Sethi, Sarah Jabbour, William F. Parker, Brett K. Beaulieu-Jones

Representation Before Training: A Practical Benchmark for Generative Medical Event Model Tokenization

Read the original on arXiv Machine Learning →

The study benchmarks tokenization choices for generative medical event models, evaluating quantization granularity, reference-range anchoring, code–value fusion, numeric and temporal encodings, and native versus harmonized event representations. Using Llama and Qwen architectures, 156 models were trained and assessed on early hospitalization data, showing that fusing codes with value deciles and using event-order or admission-relative RoPE embeddings improved predictive performance. The Common Longitudinal Intensive Care Unit Data Format (CLIF) reduced token count by 30.8% while enhancing outcomes in most families.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 8

One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models

arXiv:2602. 00541v2 Announce Type: replace Abstract: Clinical events captured in Electronic Health Records (EHR) are irregularly sampled and may consist of a mixture of discrete events and numerical measurements, such as laboratory values or treatment dosages.

By Zilin Jing, Vincent Jeanselme, Yuta Kobayashi, Simon A. Lee, Chao Pang, Aparajita Kashyap, Yanwei Li, Xinzhuo Jiang, Shalmali Joshi
More like this →
arXiv Machine Learning
Jun 24

PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models

arXiv:2606. 24102v1 Announce Type: cross Abstract: Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot directly represent events containing unseen concepts or new combinations of concepts and attributes such as numeric values.

By Lin Lawrence Guo, Adam Paul Yan, Emily Vettese, Lillian Sung
More like this →
arXiv Machine Learning
Jul 20

LLM4EHR: Aligning Clinical Time Series with Medical Event Sequences via Large Language Models

arXiv:2607. 15447v1 Announce Type: new Abstract: Recent research in clinical machine learning, focusing on outcome predictions in intensive care unit (ICU), has shifted from bespoke supervised models to foundation models, utilising modern representation learning methods.

By Jingteng Li, Alexander Capstick, Louise Rigny, Iona Biggart, Neil J Sebire, Payam Barnaghi
llmsragbenchmarks
More like this →
arXiv AI
Sep 1

INTERVenE: Temporal-Abstraction-Interval Based Transformers for Short-Horizon Medical Event Prediction

INTERVenE introduces Transformer models that use a knowledge‑based temporal abstraction (KBTA) token stream of named clinical concepts instead of raw measurements, enabling per‑token attributions to resolve directly to clinical concepts. Two variants are offered: an auto‑regressive decoder that generates future abstraction trajectories with step‑wise risk readouts, and a bidirectional encoder that jointly predicts risk and time‑to‑event in a single pass. On 57,078 MIMIC‑IV admissions, the encoder variant outperforms neural baselines with a support‑weighted AUPRC of 0.672 and AUROC of 0.901, while the decoder provides complementary token‑level risk trajectories.

By Shahar Oded, Yuval Shahar
llmsbenchmarks
More like this →