arXiv AI

Quantification of Credal Uncertainty: A Distance-Based Approach

arXiv:2603. 27270v2 Announce Type: replace Abstract: Credal sets, i.

arXiv Machine Learning
Jul 7

An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression

arXiv:2504. 18433v3 Announce Type: replace Abstract: Uncertainty quantification is crucial in machine learning, yet most (axiomatic) studies of uncertainty measures focus on classification, leaving a gap in regression settings with limited formal justification and evaluations.

By Christopher B\"ulte, Yusuf Sale, Timo L\"ohr, Paul Hofman, Gitta Kutyniok, Eyke H\"ullermeier
arXiv Machine Learning
Jul 9

Optimal Conformal Prediction under Epistemic Uncertainty

arXiv:2505. 19033v2 Announce Type: replace-cross Abstract: Conformal prediction (CP) is a widely used frequentist framework to quantify uncertainty by constructing prediction sets with user-specified marginal coverage guarantees.

By Alireza Javanmardi, Soroush H. Zargarbashi, Santo M. A. R. Thies, Willem Waegeman, Aleksandar Bojchevski, Eyke H\"ullermeier
arXiv Machine Learning
Sep 11

Local Robustness Quantification for Naive Bayes Classifiers and Generative Forests: a General Approach

The paper introduces techniques for measuring the robustness of predictions made by two generative classifiers—naive Bayes classifiers and generative forests—whose underlying models are probabilistic graphical models. Robustness is defined as the degree to which the classifier’s distribution can be perturbed without altering its prediction, with perturbations explored via epsilon‑contamination, total variation distance, and chi‑squared divergence neighborhoods. Experiments on benchmark datasets show that the computed robustness values can serve as indicators of prediction trustworthiness and are compared against other existing indicators.

By Adri\'an Detavernier, Jasper De Bock
arXiv Machine Learning
Jul 10

Robustness Quantification for Discriminative Models: a New Robustness Metric and its Application to Dynamic Classifier Selection

arXiv:2603. 23318v2 Announce Type: replace Abstract: Among the different possible strategies for evaluating the reliability of individual predictions of classifiers, robustness quantification stands out as a method that evaluates how much uncertainty a classifier could cope with before changing its prediction.

By Rodrigo F. L. Lassance, Jasper De Bock
arXiv Machine Learning
1d ago

Uncertainty-Aware Learning from Multi-Expert Interval Targets

The paper introduces a method for learning from multiple experts who provide interval labels, addressing both within‑label imprecision and between‑expert variation. It harmonizes diverse label vocabularies into a shared probabilistic space, retains individual intervals using a mixture of Beta distributions, and decomposes predictive uncertainty into components that are matched to their corresponding sources of label uncertainty. On sea‑ice concentration data, the approach achieves a 31% reduction in mean absolute error compared to hard‑label baselines and outperforms several aggregation and interval‑regression methods.

By Samira Alkaee Taleghan, Younghyun Koo, Andrew P. Barrett, Farnoush Banaei-Kashani
arXiv Machine Learning
Sep 16

Observational Multiplicity

The paper introduces the concept of observational multiplicity, where multiple probabilistic classifiers can perform similarly yet produce conflicting predictions, undermining interpretability and safety. It proposes measuring this arbitrariness through a regret metric that captures how predictions could shift with different training labels. The authors present a general method to estimate regret, show it varies across dataset groups, and discuss its use for safety via abstention and targeted data collection.

By Erin George, Deanna Needell, Berk Ustun