MVC-Bench is a new benchmark designed to evaluate the calibration of vision‑language models (VLMs) and medical VLMs (Medical‑VLMs) for medical image classification. It tests calibration across robustness to modality, backbone, and domain shift; effectiveness of calibration strategies and prompt‑tuning methods; and stability under prompt‑template and random‑seed variations. The benchmark includes eight backbones, three medical modalities (fundus imaging, histopathology, chest X‑ray), and compares post‑hoc, train‑time, and zero‑shot calibration approaches, reporting accuracy, Expected Calibration Error (ECE), Maximum Calibration Error (MCE), and Adaptive Calibration Error (ACE) over 1,638 experiments, while also proposing a Multi‑Class Margin (MCM) regularization technique that improves ECE in most settings.
By Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan, Muhammad Akhtar Munir Sujair Ibrahim, Mohamed Rafeek Mareer Ahamed, Yutong Xie, Imran Razzak, Muhammad Haris Khan
arXiv:2605. 18419v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) can couple visual perception with open-ended clinical reasoning, making them attractive for computational histopathology.
By Franciskus Xaverius Erick, Johanna Paula M\"uller, Bernhard Kainz
MVC-Bench is a calibration-focused benchmark for medical vision‑language models, evaluating how well these models express confidence across different modalities, backbones, and domain shifts. It tests robustness to modality, backbone, and domain changes, the effectiveness of calibration and prompt‑tuning strategies, and stability under prompt‑template and random‑seed variations. The benchmark includes 1638 experiments, reporting accuracy and Expected Calibration Error (ECE) along with other calibration metrics, and introduces a simple train‑time calibration method, Multi‑Class Margin (MCM) regularization, that achieves the lowest ECE in most settings.
arXiv:2606. 12590v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have achieved strong performance across medical imaging tasks, yet they remain prone to factual inconsistencies, poor visual grounding, and misalignment with clinically meaningful feedback.
By Shayan Mohammadizadehsamakosh, Pritam Sarkar, Leonid Sigal, Ali Etemad, Elham Dolatabadi
arXiv:2607. 05310v1 Announce Type: new Abstract: Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining.
By Guli Zhu, Chenwei Wu, Liyue Shen
arXiv:2604.02543v2 Announce Type: replace
Abstract: As vision-language models (VLMs) are increasingly deployed in clinical decision support, more than accuracy is required: knowing when to trust thei...
By Ji Young Byun, Young-Jin Park, Jean-Philippe Corbeil, Asma Ben Abacha