arXiv AI By Yuetian Du, Yucheng Wang, Ming Kong, Tian Liang, Qiang Long, Bingdi Chen, Qiang Zhu

Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA

Read the original on arXiv AI →

arXiv:2606. 19950v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) show great potential in medical tasks, but their elicited confidence often misaligns with actual accuracy, potentially leading to misdiagnosis or overlooking correct advice.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jun 30

IMCBench: A benchmark for multimodal LLMs in Image-grounded Medical Conversations

arXiv:2606. 28556v1 Announce Type: new Abstract: Recent advances in large language models and vision-language models have enabled reasoning over multimodal data, offering opportunities for clinical applications such as decision support and triaging.

By Maria Xenochristou, Ashutosh Joshi, Korosh Vatanparvar, Mohammad Abuzar Hashemi, Prasad Kasu, Deepak Bansal, Anchal Nema, Nivedita Wadhwa, Prashams S Jain, Rebecca Abraham, Will Kimbrough, Dilek Hakkani-Tur, Wilko Schulz-Mahlendorf
arXiv Computer Vision
Sep 25

PROVE: Proof-guided Regime-aware Operator Verification for Hallucination Detection in Medical Visual Question Answering

PROVE is a black‑box hallucination detector for medical visual question answering that tailors its verification strategy to each question’s evidential structure. It classifies questions into three regimes, activates a subset of five operators per regime, and calibrates operator importance using deterministic question‑answer features to produce a risk score. On 8048 test samples across three medical VQA benchmarks and four state‑of‑the‑art vision‑language models, PROVE achieves an AUROC of 0.821, surpassing the best baseline by 0.159 with consistent improvements across all models and datasets.

By Keyang Zhou, Siyi Li, Zhongnan Shi, Qichao Ying, Wei Tang, Zhenxing Qian
arXiv AI
Jul 29

Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases

arXiv:2607. 25933v1 Announce Type: cross Abstract: Clinical diagnostic evaluation should not only assess whether models can provide correct diagnoses, but also reflect the realities of clinical practice, including progressive disclosure of multimodal information, dynamic updating of diagnostic hypotheses, and continuous refinement of clinical reasoning.

By Rui Yang, Weihao Xuan, Yi Lin, Zhuhan Bao, Jonathan Chong Kai Liew, Matthew Yu Heng Wong, Nicol\'as Lescano, Nikita R. Paripati, Emily Ling-Lin Pai, Jiarui Liu, Heli Qi, Heng-Jui Chang, Benny Kai Guo Loo, Huitao Li, Kunyu Yu, Yufan Wang, Chuan Hong, Shijian Lu, Douglas Teodoro, Naoto Yokoya, Ross Koppel, Mona Diab, Hua Xu, David W. Bates, Nan Liu, Yifan Peng