arXiv Machine Learning

Evaluating utility of clinical prediction models: A comparison of decision curve analysis and cost curves

arXiv Statistics ML
5d ago

Cost-based variable selection for prediction models

The article introduces a decision‑analytic variable selection algorithm that optimizes clinical prediction models by balancing predictive performance against the costs and harms of measuring variables. It exhaustively compares net benefit across all predictor combinations, with a backward‑elimination extension for high‑dimensional data, and selects variables only when their added predictive value outweighs their measurement costs. The method, implemented in the R package NBvarsel, was demonstrated on simulated data and three real case studies, producing parsimonious models that maintained discrimination and calibration while reducing unnecessary clinical burden.

By Lasai Barre\~nada, Andrew Vickers, Laure Wynants, Bieke Tack, Dirk Timmerman, Ewout Steyerberg, Ben Van Calster
arXiv Machine Learning
Sep 18

Interpretable and Calibrated Classification of Clinical Data Using Supervised Feature Binarization

The paper introduces a statistically grounded framework for interpretable, rule-based clinical classification using Bernoulli Naïve Bayes (BNB). It employs supervised chi‑square‑guided binarization to convert continuous medical variables into binary indicators, enabling BNB to handle continuous data while maintaining transparency. On three benchmark datasets—Pima Indians Diabetes, Wisconsin Breast Cancer, and Heart Failure Prediction—the method achieved AUCs of 0.800, 0.984, and 0.919, respectively, and demonstrated reliable probability calibration through cross‑validated analysis and post‑hoc beta calibration.

By Antony Garcia, Adrian Noriega, Gabrielle Britton, Xinming Huang
arXiv AI
Sep 3

The Ceiling Is in the Channel: Auditing Learner Gaps and Measurement Frontiers in Clinical Prediction

The paper introduces a framework that distinguishes two causes of saturation in clinical prediction: a learner gap, where the model fails to use available information, and a measurement‑channel ceiling, where the recorded variables limit performance. It provides theoretical characterizations, finite‑sample diagnostics, and empirical audits across three large cohorts, showing that well‑tuned models approach the frontier while deficient learners leave large gaps. A PRISMA‑guided synthesis across 104 tasks reveals consistent channel‑level patterns, suggesting that improving the learner or the measurement channel can audit and potentially lift performance.

By Sayeed Shafayet Chowdhury, Nusrat Jahan, Snehasis Mukhopadhyay, Shiaofen Fang, Vijay R. Ramakrishnan
arXiv Machine Learning
Sep 14

Optimizing for the decision not the prediction: an exploration of Smooth Net Benefit as a training objective

The study introduces Smooth Net Benefit (σNB), a differentiable approximation of Net Benefit, as a training objective aimed at aligning predictive models with threshold‑specific clinical decisions. Experiments on the Framingham cardiovascular risk dataset and 44 TabZilla datasets show that σNB training yields modest improvements for logistic regression but little to no benefit for more flexible models such as GAMs and XGBoost. The authors conclude that σNB is not a universal replacement for negative log‑likelihood training, though it may be worth exploring in contexts where model flexibility is limited.

By Koen M. F. Gorgels, Lasai Barre\~nada, Maarten van Smeden, Ben Van Calster, Ewout W. Steyerberg, Wouter A. C. van Amsterdam
arXiv AI
5d ago

MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine

MedJev is an open System One model that answers medical questions in a single forward pass without generating text, enabling fast, local clinical decision support. Across ten medical benchmarks, MedJev achieves competitive accuracies—72.5 % on MedQA, 75.4 % on USMLE, and 80.1 % on NEJM—while maintaining well‑calibrated confidence estimates. On patient‑level tasks, it attains an AUROC of 0.926 for in‑hospital mortality, surpassing XGBoost, and introduces a lightweight two‑parameter recalibration method that reduces miscalibration from 0.286 to 0.007 using only 50–200 labeled examples.

By Simon A. Lee, Jiahang Sha, Moonseong Jeong, Yunbei Pan, Jeffrey N. Chiang
arXiv AI
Aug 7

From Continuous Predictors to Clinical Thresholds: Early Evidence on Performance Trade-offs of Guideline-Based Categorisation for Ischaemic Stroke Outcome Prediction

arXiv:2608. 05203v1 Announce Type: new Abstract: Machine learning models achieve strong predictive accuracy for 90-day outcome prediction in acute ischaemic stroke, yet clinical adoption is limited by the misalignment of model explanations with clinicians' reasoning.

By Esra Zihni, Katryna Cisek, Hamzah Ziadeh, Hendrik Knoche, Robert Mikulik, John D. Kelleher