The paper introduces MOVAL, a model evaluation tool that estimates performance of deep neural networks on unlabeled test data by calibrating class-wise confidence scores. MOVAL mitigates issues from distributional shifts and class imbalance, enabling estimation of metrics beyond accuracy such as F1-Score and AUC. Experiments on natural and medical image datasets demonstrate state‑of‑the‑art results across classification and segmentation tasks, providing accurate class, instance, and pixel‑level performance estimates for safe deployment.
By Zeju Li, Konstantinos Kamnitsas, Mobarakol Islam, Melanie Roschewitz, Cheng Ouyang, Chen Chen, Ben Glocker
arXiv:2610.03142v1 Announce Type: new
Abstract: Deep neural networks remain vulnerable to adversarial perturbations, which can distort not only predictions but also confidence scores, undermining unc...
By Leo Fillioux, Stergios Christodoulidis, Stergios Christodoulidis, Maria Vakalopoulou, Jose Dolz
arXiv:2605. 30188v2 Announce Type: replace-cross Abstract: Reliable probability estimates are critical in many machine learning applications, yet modern classifiers are often poorly calibrated.
By Eug\`ene Berta, David Holzm\"uller, Francis Bach, Michael I. Jordan
arXiv:2608. 12100v1 Announce Type: cross Abstract: In high-stakes applications, reliable confidence estimates are as important as the predictions themselves.
By Coby Penso
arXiv:2601. 07965v2 Announce Type: replace Abstract: When a model knows when it does not know, many possibilities emerge.
By Chenjie Hao, Weyl Lu, Yuko Ishiwaka, Zengyi Li, Weier Wan, Yubei Chen
arXiv:2608. 09768v1 Announce Type: new Abstract: A prediction that is both confident and wrong is a critical reliability failure because it can bypass abstention and human review precisely when the model is mistaken.
By Ange-Cl\'ement Akazan, Ineza Remy Mugenga, Abebe Geletu, Jean Medard Ngnotchouye, Issa Karambal
arXiv:2606. 16196v1 Announce Type: new Abstract: Deep neural networks have achieved remarkable performance across medical imaging tasks, yet their tendency to overgeneralize under distributional shifts poses a major obstacle to safe clinical deployment.
By Anju Chhetri, Pratik Shrestha, Ramesh Rana, Prashnna Gyawali, Binod Bhattarai
arXiv:2606. 16883v1 Announce Type: cross Abstract: Generalization is a critical property of data-driven models, particularly deep learning models deployed in safety-critical applications.
By Abdul-Rauf Nuhu, Parham M. Kebria, Vahid Hemmati, Mahmoud N. Mahmoud, Edward Tunstel, Abdollah Homaifar
arXiv:2606. 28654v1 Announce Type: cross Abstract: Deep Neural Network (DNN) classifiers suffer from poor calibration when their softmax outputs (predictive confidence) deviate from the empirical likelihoods.
By Thiru Thillai Nadarasar Bahavan, Sachith Seneviratne, Saman Halgamuge
TestifAI is a deep learning testing framework that estimates model robustness against combinations of input perturbations efficiently. It allows users to define structured spaces of semantic perturbations and severity levels, then query robustness for any combination. By using partial model tomography, TestifAI reconstructs higher-order perturbation effects from low-order tests, achieving less than 7% error while reducing inferences by 60–80%.
The paper introduces Conflict‑Aware Evidential Deep Learning (C‑EDL), a lightweight post‑hoc method that improves uncertainty quantification for deep learning models. C‑EDL applies diverse, task‑preserving transformations to each input and uses representational disagreement to adjust predictions, thereby reducing overconfident errors on adversarial and out‑of‑distribution data. Experiments demonstrate that C‑EDL outperforms existing Evidential Deep Learning variants and baselines, achieving up to 55 % reduction in coverage for OOD data and 90 % for adversarial data across multiple datasets and attack types.
By Charmaine Barker, Daniel Bethell, Simos Gerasimou
arXiv:2509. 07605v2 Announce Type: replace-cross Abstract: Class imbalance poses a significant challenge to supervised classification, particularly in critical domains like medical diagnostics and anomaly detection where minority class instances are rare.
By Ali Nawaz, Amir Ahmad, Shehroz S. Khan