arXiv AI

When uncertainty quantification fails: structural vulnerabilities and corrections for selective prediction under clinical class imbalance

arXiv Machine Learning
Jul 30

Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

arXiv:2607. 27143v1 Announce Type: new Abstract: High-stakes decision systems in credit scoring, fraud detection, healthcare, and industrial safety require reliable uncertainty quantification under severe class imbalance and asymmetric error costs.

By Manpreet Singh, Akshatha Srikantha, Shyamal Lakhanpal
arXiv AI
Sep 3

The Ceiling Is in the Channel: Auditing Learner Gaps and Measurement Frontiers in Clinical Prediction

The paper introduces a framework that distinguishes two causes of saturation in clinical prediction: a learner gap, where the model fails to use available information, and a measurement‑channel ceiling, where the recorded variables limit performance. It provides theoretical characterizations, finite‑sample diagnostics, and empirical audits across three large cohorts, showing that well‑tuned models approach the frontier while deficient learners leave large gaps. A PRISMA‑guided synthesis across 104 tasks reveals consistent channel‑level patterns, suggesting that improving the learner or the measurement channel can audit and potentially lift performance.

By Sayeed Shafayet Chowdhury, Nusrat Jahan, Snehasis Mukhopadhyay, Shiaofen Fang, Vijay R. Ramakrishnan
arXiv Machine Learning
6d ago

Fragility-aware Classification for Risk Characterization and Robust Learning

The paper introduces the Fragility Index (FI), a new performance metric that measures the tail risk of confident misclassifications in classification models. FI is framed within a robust satisficing framework to assess classifier sensitivity under distributional uncertainty, and the authors develop a surrogate-based training method with provable bounds linking training fragility to ranking-error FI. The approach is extended to common loss functions and includes an FI-based regularizer for deep neural networks, with empirical studies on medical diagnosis tasks showing that FI reveals risk not captured by accuracy or AUC and leads to models with lower operational costs in a human–AI diagnostic workflow.

By Zheng Cui, Daniel Zhuoyu Long, Jin Qi, Chen Yang, Ruohan Zhan
arXiv Machine Learning
Jul 14

Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts

arXiv:2607. 11656v1 Announce Type: cross Abstract: Accurate diagnostic classification and disease-severity prediction for Alzheimer's disease are hampered by the incompleteness and heterogeneity of real-world clinical data.

By Christelle Schneuwly Diaz, Narmina Baghirova, Duy-Thanh Vu, Duy-Cat Can, Gilles Allali, Philippe Ryvlin, Oliver Y. Ch\'en
arXiv AI
Sep 15

Adaptive Conformal Redistribution for Inter-class Transitional Uncertainty in Medical Image Classification

The paper introduces AdaConRed, a label‑free post‑conformal decision rule that transforms ambiguous conformal prediction sets into single class assignments for medical image classification. It employs a five‑stage pipeline—vision‑language generative augmentation, a frozen DermFoundation encoder, a lightweight MLP classifier, an entropy‑modulated margin‑aware nonconformity score, and a reassignment step for transitional samples—to improve accuracy on OSCC and ISIC benchmarks. Results show notable gains in malignant class accuracy while maintaining overall performance.

By Saibal Ghosh, Samarup Bhattacharya, Sanjoy Kumar Saha, Umapada Pal, Tapabrata Chakraborti
arXiv AI
Aug 20

FairGlucose: A CGM Fairness Benchmark Reveals Subgroup Disparities Hidden in Population-Level Validation

FairGlucose is a 300‑patient CGM cohort balanced across 12 demographic strata, providing 132,480 forecasting samples and 3,945 behavioral events. Benchmarking 33 models on 2‑hour glucose forecasting revealed that population‑level validation masks significant subgroup disparities, with error ratios ranging from 0.8 to 1.4 and T1D patients experiencing 6 mg/dL higher error than T2D. The study shows that these gaps persist across all models, align with clinically hard cases, and vary with input‑length sensitivity, underscoring the need for subgroup‑disaggregated reporting in digital health AI.

By Junjie Luo, Xuzhe Zhi, Rui Han, Abhimanyu Kumbara, Anand K. Iyer, Mansur E. Shomali, Ritu Agarwal, Guodong Gordon Gao
arXiv Machine Learning
Sep 18

Machine-Learning Assessment of the Predictive Value of Inflammatory Biomarkers for Cognitive Impairment in an Older Hispanic Adult Cohort

The study evaluates whether inflammatory biomarkers can predict cognitive impairment in older Hispanic adults using interpretable machine learning on a small clinical dataset. A leakage‑safe Bernoulli/Categorical Naive Bayes model was trained on 165 participants from the Panama Aging Research Initiative, with continuous predictors discretized via supervised chi‑square and income treated categorically. The biomarker I‑309 (CCL1) emerged as the sole reliable incremental predictor, boosting ROC‑AUC from 0.630 to 0.740 and achieving statistically significant performance across repeated cross‑validation and random partitions.

By Antony Garcia, Gabrielle Britton, Alcibiades Villarreal, Diana Oviedo, Giselle Rangel, Xinming Huang
arXiv Machine Learning
Oct 22

Extending ROC Analysis to Uncertainty-Aware Risk Prediction with an Interval-Based AUC (iAUC)

The paper introduces an uncertainty‑aware ROC framework and an interval‑based AUC (iAUC) metric for evaluating interval‑valued risk predictions. It constructs two ROC‑style curves yielding “AUC_L” and “AUC_U”, which decompose rankings into confidently correct, ambiguous, and confidently incorrect regions. The framework links interval coverage to theoretical bounds on optimal AUC and is applicable to Bayesian, bootstrap, and ensemble uncertainty methods, with experiments on clinical data and MIMIC‑IV mortality prediction demonstrating its utility for model comparison and risk‑aware selection.

By Yuqi Li, Matthew M. Engelhard
arXiv AI
Aug 11

FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records

arXiv:2505. 16941v4 Announce Type: replace-cross Abstract: Foundation models (FMs) promise to address core limitations of traditional supervised machine learning: (i) reliance on large amounts of labeled data, (ii) task specificity, and (iii) poor transportability.

By Vincent Jeanselme, Zilin Jing, Aparajita Kashyap, Chao Pang, Florent Pollet, Young Sang Choi, Xinzhuo Jiang, Yuta Kobayashi, Yanwei Li, Sara Matijevic, Karthik Natarajan, Shalmali Joshi