arXiv AI June 26, 2026 By Annika Schneider, Tommy Rochussen, Joshua Stiller, Vincent Fortuin

Decision-Aligned Evaluation of Uncertainty Quantification

arXiv:2606. 26990v1 Announce Type: cross Abstract: Uncertainty estimates in machine learning are typically evaluated using generic metrics such as the negative log-likelihood and expected calibration error, yet good performance on such metrics does not necessarily imply high utility in downstream decisions.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 17

Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning

arXiv:2607. 14817v1 Announce Type: cross Abstract: Current evaluation of epistemic uncertainty relies on tasks such as out-ofdistribution detection and active learning.

By Jakub Paplh\'am, Willem Waegeman, Eyke H\"ullermeier, Vojt\v{e}ch Franc

benchmarks

arXiv Machine Learning

Jul 7

An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression

arXiv:2504. 18433v3 Announce Type: replace Abstract: Uncertainty quantification is crucial in machine learning, yet most (axiomatic) studies of uncertainty measures focus on classification, leaving a gap in regression settings with limited formal justification and evaluations.

By Christopher B\"ulte, Yusuf Sale, Timo L\"ohr, Paul Hofman, Gitta Kutyniok, Eyke H\"ullermeier

arXiv Machine Learning

Jun 19

On the QUEST for Uncertainty Quantification via Highest Density Regions

arXiv:2606. 19569v1 Announce Type: new Abstract: Uncertainty quantification (UQ) is essential for reliable decision-making in safety-critical applications in probabilistic machine learning.

By Sam Goring, Tom Kuipers, Nicola Paoletti, David S. Watson

benchmarks safety

arXiv Machine Learning

Jun 10

Can we trust our models? Epistemic calibration in second-order classification

arXiv:2606. 10777v1 Announce Type: new Abstract: Uncertainty estimation is critical for deploying machine learning models in high-stakes settings.

By Arthur Hoarau

arXiv Machine Learning

Jul 30

Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

arXiv:2607. 27143v1 Announce Type: new Abstract: High-stakes decision systems in credit scoring, fraud detection, healthcare, and industrial safety require reliable uncertainty quantification under severe class imbalance and asymmetric error costs.

By Manpreet Singh, Akshatha Srikantha, Shyamal Lakhanpal

benchmarks safety

arXiv Machine Learning

Jun 8

Uncertainty-Guided Label Rebalancing for CPS Safety Monitoring

arXiv:2603. 25670v3 Announce Type: replace Abstract: Safety monitoring is essential for Cyber-Physical Systems (CPSs).

By John Ayotunde, Qinghua Xu, Guancheng Wang, Lionel C. Briand

benchmarks safety

Related stories

Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning

An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression

On the QUEST for Uncertainty Quantification via Highest Density Regions

Can we trust our models? Epistemic calibration in second-order classification

Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

Uncertainty-Guided Label Rebalancing for CPS Safety Monitoring