The paper introduces the Fragility Index (FI), a new performance metric that measures the tail risk of confident misclassifications in classification models. FI is framed within a robust satisficing framework to assess classifier sensitivity under distributional uncertainty, and the authors develop a surrogate-based training method with provable bounds linking training fragility to ranking-error FI. The approach is extended to common loss functions and includes an FI-based regularizer for deep neural networks, with empirical studies on medical diagnosis tasks showing that FI reveals risk not captured by accuracy or AUC and leads to models with lower operational costs in a human–AI diagnostic workflow.
arXiv:2608. 09768v1 Announce Type: new Abstract: A prediction that is both confident and wrong is a critical reliability failure because it can bypass abstention and human review precisely when the model is mistaken.
The paper investigates the consistency of surrogate loss methods for classification and policy learning when the set of admissible classifiers is constrained, such as by interpretability or fairness requirements. It shows that hinge loss is the only surrogate that preserves consistency when constraints limit only the prediction set, but consistency can fail if constraints also restrict the functional form. The authors derive conditions guaranteeing consistency for hinge-risk-minimizing classifiers and use these results to design efficient hinge-loss-based procedures for monotone classification problems.
arXiv:2602.04775v2 Announce Type: replace Abstract: In high-stakes risk prediction, interval-valued predictions provide a natural way to represent predictive uncertainty. However, standard evaluation...
The paper introduces a method for learning risk scores that remain reliable even when historical data contain unobserved confounders. By treating propensity weights as uncertain and applying sensitivity analysis with Wasserstein distributionally robust optimization, the authors formulate a robust learning problem solvable via an exponential cone program. Experiments on semi‑synthetic UCI data show the approach improves calibration by up to 29.2% over traditional benchmarks and 11.1% over the state of the art, without harming other performance metrics.
arXiv:2607. 03075v1 Announce Type: new Abstract: Safety-critical applications require classifiers that are both robust and reliable.
In high-stakes healthcare applications, machine learning models are frequently trained on data from one patient population and deployed on another, creating a distribution shift that degrades both acc...
arXiv:2606. 01746v1 Announce Type: cross Abstract: Modern neural networks are highly susceptible to adversarial perturbations.
arXiv:2607. 16802v1 Announce Type: new Abstract: Deep survival models are evaluated almost exclusively by the concordance index (C-index), yet they are commonly trained using likelihood objectives such as the Cox partial likelihood, discrete-time negative log-likelihood, and DeepHit likelihood.
arXiv:2511. 22823v2 Announce Type: replace-cross Abstract: Weakly supervised learning has emerged as a practical alternative to fully supervised learning when complete and accurate labels are costly or infeasible to acquire.
arXiv:2607. 27143v1 Announce Type: new Abstract: High-stakes decision systems in credit scoring, fraud detection, healthcare, and industrial safety require reliable uncertainty quantification under severe class imbalance and asymmetric error costs.
arXiv:2602. 21160v3 Announce Type: replace-cross Abstract: In safety-critical classification, the cost of failure is often asymmetric, yet Bayesian deep learning summarises epistemic uncertainty with a single scalar, mutual information (MI), that cannot distinguish whether a model's ignorance involves a benign or safety-critical class.
The paper introduces V-ECE, a variational estimator for general Expected Calibration Errors (ECE) that avoids binning and works with any convex divergence, including Lp distances. By using prediction‑dependent proper scores and a residual‑fitted temperature scaling via gradient boosting, V‑ECE provides a lower‑bound estimate of true calibration error. Experiments on semi‑synthetic benchmarks show V‑ECE outperforms existing binary and multiclass estimators, with additional theoretical insights on bias and confidence estimation.