arXiv AI By Jize Li

When Does Machine Learning Beat Value Sorting? A Three-Dataset Diagnostic of Exposure-Weighted Shipment Prioritization

Read the original on arXiv AI →

arXiv:2607. 18573v1 Announce Type: new Abstract: Delay-risk models are usually judged by predictive accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 28

Diagnosing Conformal Prediction Failures Under Distribution Shift: A COVID-19 Case Study

The paper introduces SHAP concentration as a pre‑deployment diagnostic for detecting when conformal prediction will fail under distribution shift, specifically in gradient‑boosted classifiers. Using a COVID‑19 supply‑chain case study, the authors show that higher feature‑importance concentration correlates with larger drops in coverage, while standard shift detectors cannot differentiate between catastrophic and robust outcomes. The diagnostic is validated on additional datasets, and a formal theorem links concentration to worsening conformity‑score bounds, though it does not capture global‑sensitivity failures in neural networks.

By Chorok Lee
arXiv Machine Learning
1d ago

The Hidden Costs of 99% Accuracy: A Trustworthiness Audit of the Telco Customer Churn Benchmark

The paper audits the IBM Telco Customer Churn benchmark, revealing that common practices inflate performance metrics. It shows that pre‑split SMOTE boosts churn‑class F1 by 13.1 points, that isotonic regression is the best calibration method while temperature scaling fails on tree ensembles, and that the cost‑optimal decision threshold is 5–10 times lower than the F1‑optimal one, saving about $77,000 per 1,000 customers. The authors also test generalisation on Iranian Telecom and Bank churn datasets, and propose a four‑component reporting checklist with reproducible code.

By Soumyadeep Roy