arXiv Machine Learning

Alert-List Concordance for Evaluating Imputation Strategies in ICU Sepsis Prediction

The paper introduces alert‑list concordance, a metric that evaluates whether different imputation strategies for ICU sepsis prediction models identify the same high‑risk patients, measured by Jaccard similarity of top‑k% alert sets with bootstrap confidence intervals. Using PhysioNet/CinC 2019 data, the authors compare nine imputation methods and a native‑missingness baseline, finding that many methods have similar AUROC yet produce alert lists that overlap only 51–75% of the time. ForwardFill stands out as an outlier, achieving higher discrimination but lower overlap (40–45%).

arXiv AI
Aug 28

Standalone LLM and a Pre-specified Agentic Pipeline for Explaining ICU Mortality Predictions: a Feasibility Study on the eICU Demo Dataset

The study evaluates a standalone large language model (LLM) versus a four‑step agentic pipeline for generating explanations of ICU mortality predictions on the eICU Demo dataset. XGBoost achieved an AUROC of 0.855 and an AUPRC of 0.332. In a 38‑case explanation subset, the standalone LLM produced one explanation with outcome leakage, while the agentic pipeline produced none; among 14 overlapping SHAP cases, the standalone LLM had higher SHAP alignment and direction consistency, whereas the agentic pipeline showed better guideline grounding, value specificity, and plausibility.

By Di Zhu, Chen Xie, Haoyun Zhang, Zihan Wei, Ziwei Wang, Jiazhao Shi, Ziyu Wang, Qiyang Xie
arXiv Machine Learning
Oct 12

Hidden Data Challenges in Clinical Machine Learning

The paper highlights hidden data challenges in clinical machine learning, focusing on how commonly reused preprocessing pipelines for electronic health records embed assumptions about data availability, episode definitions, and missingness. Using MIMIC-IV, the authors identify violations in temporal, semantic, and population alignment—leading to temporal leakage, feature-task misalignment, and biased imputation—and show that these ambiguities can alter model signals without affecting overall predictive performance. They provide practical recommendations and open-source code for more temporally valid, clinically aligned, and reproducible preprocessing.

By Carolin Ciss\'ee, Dennis Fast, Raneen Younis, Alexander L\"oser, Wolfgang Nejdl, Zahra Ahmadi
arXiv Machine Learning
Sep 29

Missingness-Aware Conformal Prediction Under Cross-Hospital Distribution Shift

The paper introduces a missingness‑aware conformal calibration method for mortality prediction that accounts for cross‑hospital distribution shifts. By selecting a measurement on an independent sample, grouping patients by whether that measurement is recorded, and applying Mondrian calibration within each group, the method avoids reusing calibration outcomes. Experiments on eICU and MIMIC‑IV data show that, compared to pooled calibration, it reduces the worst‑group coverage gap by a median of 1.9 percentage points across six settings, though the benefit varies with predictor and hospital.

By Liang You, Dongwen Ou, Hengyu Shi, Siyuan Dai
arXiv AI
Oct 31

NECA: Neighbor Event Curve Augmentation for Multi-Horizon ICU Deterioration Prediction

The paper introduces NECA, a model‑agnostic augmentation technique that retrieves similar patients from the training set and transforms their organ‑dysfunction event onset times into cumulative event‑curve features for multiple future horizons (6h, 12h, 24h, 42h). These features are concatenated with the original patient representation and fed to standard downstream predictors, leading to improved PR‑AUC and ROC‑AUC on PhysioNet 2012 and a MIMIC‑IV subset across several organ‑dysfunction tasks. Ablation studies show that clinically meaningful retrieval and horizon‑specific neighbor risk are the main drivers of performance gains, especially for neurological dysfunction events.

By Chenyang Wang, Jiawei Theresa Tang, Xinran Liu, Xinyu Lu, Xulei Yang
Hugging Face Trending Papers
5d ago

Where Is a Prediction Model Miscalibrated, and for Whom? Local Audits in Outcome-Learned Pockets

The paper introduces a local audit method that identifies where a prediction model is miscalibrated and for whom, by grouping similar observations into ‘pockets’ using random‑forest similarity and ridge‑regression. The audit, which requires only features, outcomes, and fixed predictions, demonstrates superior detection of group‑specific errors in simulations and real synthetic data, revealing nuanced over‑ and under‑prediction patterns that overall metrics like AUC can miss.

arXiv AI
Aug 28

Learning a Continuous Sepsis Severity Score Without Hour-by-Hour Supervision: A Two-Site Retrospective Study

The study presents a new sepsis severity score derived from 43 routinely charted variables over a 72‑hour window, trained on 29,116 and 7,691 adult patients from two Massachusetts hospitals. Using mortality as a treatment‑level ranking signal, the index assigns higher scores to non‑survivors (1.19–1.64 points higher on a 0–10 scale) across various baseline strata and shows significant correlations with lactate, MAP, and creatinine changes. Cross‑institutional agreement and external validation demonstrate robust performance, suggesting the score could serve as a decision‑support tool alongside clinical judgment.

By Kevin Zhu, Ryan Zhang, Baraa Abed, Tilendra Choudhary, Malvern Madondo, Mehak Arora, Yixuan Yang, Alasdair Gent, Aditya Nagori, Omer T. Inan, Krista L. Haines, Patrick Georgoff, Suresh M. Agarwal, Vijay Krishnamoorthy, Tetsu Ohnuma, Mihai V. Podgoreanu, Michael R. Pinsky, Gilles Clermont, Craig M. Coopersmith, Craig S. Jabaley, Rishikesan Kamaleswaran