arXiv Computer Vision

A Decision-Aware Evaluation Protocol for Uncertainty in Medical Image Segmentation

The paper introduces a decision‑aware evaluation protocol for medical image segmentation uncertainty that separates uncertainty estimation from deferral policy and assesses their joint performance under fixed review budgets. It reports metrics such as fixed‑budget error reduction, risk‑coverage behavior, region‑level review proxies, structural failure modes, and sensitivity to review assumptions. Experiments on retinal vessel, skin lesion, and polyp segmentation with various backbones and uncertainty estimators show that estimator rankings vary with review unit, uncertainty target, and cost model, and that boundary and thin‑structure failures are partly due to annotation ambiguity.

arXiv Machine Learning
Jul 28

Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation

arXiv:2607. 22727v1 Announce Type: cross Abstract: Medical image segmentation models often report high benchmark accuracy under ideal imaging conditions, yet their failures under clinical degradation can be quiet: sensor noise, patient motion, low- resolution acquisition, and contrast variability may all alter model behavior without producing an obvious warning.

By Pranav Kaliaperumal, Manisha Kaliaperumal
arXiv Machine Learning
Aug 18

Beyond Boundary Noise: Aggregated Aleatoric Uncertainty Fails to Capture Presence Ambiguity in 3D Lung Nodule Segmentation

arXiv:2608. 14766v1 Announce Type: cross Abstract: Uncertainty estimation is critical for the safe clinical deployment of deep learning in medical image segmentation, with aleatoric uncertainty theoretically designed to capture irreducible data ambiguity.

By Simon Baur, Arne Schernich, Ekin B\"oke, Wojciech Samek, Jackie Ma
arXiv Computer Vision
Aug 27

Reliability analysis for BraTS-GoAT segmentation: a controlled robustness study of deep-ensemble uncertainty

The study evaluates the reliability of deep‑ensemble uncertainty for brain tumour segmentation on the BraTS‑GoAT dataset. A 5‑fold cross‑validated nnU‑Net baseline and a 3‑seed deep ensemble were compared for calibration and error detection; the ensemble showed modest gains in calibration on in‑distribution data but the single model’s confidence remained flat while accuracy degraded under synthetic corruptions. Disagreement among ensemble members rose sharply with corruption severity, proving to be a more sensitive indicator of acquisition shift than single‑model confidence.

By Riya Deepak Shet, Chenxi Liang, Le Zhang
Hugging Face Trending Papers
Aug 27

MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

MVC-Bench is a calibration-focused benchmark for medical vision‑language models, evaluating how well these models express confidence across different modalities, backbones, and domain shifts. It tests robustness to modality, backbone, and domain changes, the effectiveness of calibration and prompt‑tuning strategies, and stability under prompt‑template and random‑seed variations. The benchmark includes 1638 experiments, reporting accuracy and Expected Calibration Error (ECE) along with other calibration metrics, and introduces a simple train‑time calibration method, Multi‑Class Margin (MCM) regularization, that achieves the lowest ECE in most settings.

arXiv Computer Vision
Aug 28

MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

MVC-Bench is a new benchmark designed to evaluate the calibration of vision‑language models (VLMs) and medical VLMs (Medical‑VLMs) for medical image classification. It tests calibration across robustness to modality, backbone, and domain shift; effectiveness of calibration strategies and prompt‑tuning methods; and stability under prompt‑template and random‑seed variations. The benchmark includes eight backbones, three medical modalities (fundus imaging, histopathology, chest X‑ray), and compares post‑hoc, train‑time, and zero‑shot calibration approaches, reporting accuracy, Expected Calibration Error (ECE), Maximum Calibration Error (MCE), and Adaptive Calibration Error (ACE) over 1,638 experiments, while also proposing a Multi‑Class Margin (MCM) regularization technique that improves ECE in most settings.

By Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan, Muhammad Akhtar Munir Sujair Ibrahim, Mohamed Rafeek Mareer Ahamed, Yutong Xie, Imran Razzak, Muhammad Haris Khan
arXiv AI
Oct 5

Uncertainty as a Proxy for Semantic Correctness in Diffusion-Based Medical Image Synthesis

The paper investigates whether uncertainty can act as a proxy for semantic correctness in diffusion-based medical image synthesis, specifically for generating contrast‑enhanced CT (CECT) from non‑contrast CT (NCCT). Using the AortaDiff framework, which produces both CECT images and lumen segmentations, the authors compare six uncertainty estimation methods across pixel, region, and image levels, including their ability to detect out‑of‑distribution cases. They find that uncertainty is informative at all spatial scales, remains useful under distribution shift, and that MCDropout, in particular, offers reliable quality filtering with no extra training cost.

By Yuxuan Ou, Konstantinos Kamnitsas, OxAAA Study, AICT Consortium, Regent Lee, Vicente Grau