MVC-Bench is a new benchmark designed to evaluate the calibration of vision‑language models (VLMs) and medical VLMs (Medical‑VLMs) for medical image classification. It tests calibration across robustness to modality, backbone, and domain shift; effectiveness of calibration strategies and prompt‑tuning methods; and stability under prompt‑template and random‑seed variations. The benchmark includes eight backbones, three medical modalities (fundus imaging, histopathology, chest X‑ray), and compares post‑hoc, train‑time, and zero‑shot calibration approaches, reporting accuracy, Expected Calibration Error (ECE), Maximum Calibration Error (MCE), and Adaptive Calibration Error (ACE) over 1,638 experiments, while also proposing a Multi‑Class Margin (MCM) regularization technique that improves ECE in most settings.
By Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan, Muhammad Akhtar Munir Sujair Ibrahim, Mohamed Rafeek Mareer Ahamed, Yutong Xie, Imran Razzak, Muhammad Haris Khan
arXiv:2606. 27023v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) applied to Medical Visual Question Answering (VQA) tend to produce overconfident outputs regardless of actual correctness, and existing verbalized confidence calibration methods, developed primarily for text only LLMs, do not account for the multimodal nature of medical image understanding.
By Eren Senoglu, Federico Toschi, Nicolo Brunello, Andrea Sassella, Mark James Carman
The paper introduces a decision‑aware evaluation protocol for medical image segmentation uncertainty that separates uncertainty estimation from deferral policy and assesses their joint performance under fixed review budgets. It reports metrics such as fixed‑budget error reduction, risk‑coverage behavior, region‑level review proxies, structural failure modes, and sensitivity to review assumptions. Experiments on retinal vessel, skin lesion, and polyp segmentation with various backbones and uncertainty estimators show that estimator rankings vary with review unit, uncertainty target, and cost model, and that boundary and thin‑structure failures are partly due to annotation ambiguity.
By Saket Maganti, Shreyank N Gowda
arXiv:2607. 01902v1 Announce Type: cross Abstract: Reliable confidence estimates are essential in semantic segmentation, especially in safety-critical settings where overconfident errors can mislead downstream decisions.
By Tristan Kirscher (ICube), Kim-Celine Kahl (DKFZ), Balint Kovacs (DKFZ), Maximilian R. Rokuss (DKFZ), Klaus Maier-Hein (DKFZ), Xavier Coubez (ICube), Philippe Meyer (ICube), Sylvain Faisan (ICube)
Reliable confidence estimates are essential in semantic segmentation, especially in safety-critical settings where overconfident errors can mislead downstream decisions. Yet modern segmentation models often remain miscalibrated.
arXiv:2608.21300v1 Announce Type: new
Abstract: Foundation models for medical image segmentation, like prompt-based MedSAM, generalize well across domains and modalities, often in zero or few-shot se...
By Marko Haralovi\'c, Sounic Akkaraju, Carlo Baretta, Vasil Zapryanov, Alexia Briassouli
arXiv:2606. 23487v2 Announce Type: replace Abstract: Medical vision-language models (VLMs) such as BiomedCLIP generalize broadly, but adapting them to a clinical service is as much a safety problem as an accuracy one.
By Rishabh Jha, Amrita Singh, Prashanna Chudal
The paper investigates how pre‑training strategy, dataset size, and domain affect uncertainty estimation in vision medical foundation models. It compares point‑prediction calibration with conformal (region) prediction across retinal, histopathological, and chest X‑ray models, finding that domain‑specific, self‑supervised pre‑training yields better calibration and more efficient conformal sets. The study shows that standard recalibration alone cannot fully reconcile uncertainty differences between models trained on different data sources.
By Haoxu Huang, Narges Razavian
arXiv:2605. 18419v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) can couple visual perception with open-ended clinical reasoning, making them attractive for computational histopathology.
By Franciskus Xaverius Erick, Johanna Paula M\"uller, Bernhard Kainz
arXiv:2606. 15910v2 Announce Type: replace Abstract: A vision-language model can answer a question about a chest radiograph or a pathology slide fluently and confidently while barely using the image, relying instead on language priors.
By Reza Khanmohammadi, Kundan Thind, Mohammad M. Ghassemi
arXiv:2609.39266v1 Announce Type: new
Abstract: Fine-grained vision-language alignment in chest radiography enables zero-shot classification, grounding, and segmentation without task-specific annotat...
By Qixing Zhao, Jinpeng Li
arXiv:2607. 05310v1 Announce Type: new Abstract: Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining.
By Guli Zhu, Chenwei Wu, Liyue Shen