arXiv Computation and Language September 4, 2026 By Arnisa Fazla, Alberto Testoni, Ameen Abu-Hanna, Barbara Plank, Iacer Calixto

Uncertainty Is Not a Safety Net for Clinical VQA, but Can It Anticipate Model Failure?

Read the original on arXiv Computation and Language →

The paper evaluates uncertainty estimation (UE) methods for clinical vision‑language models (VLMs) on visual question answering (VQA). Across 8 UE techniques and 12 VLMs, UE quality tracks model accuracy, degrading where performance is weakest, and fails to signal uncertainty when models are stressed by hiding the correct answer (NOTA perturbations). However, UE on unperturbed inputs reliably predicts which predictions will collapse under NOTA, suggesting UE can diagnose model fragility.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

3d ago

Rethinking Correctness for Uncertainty Estimation in Clinical Prediction with Vision-Language Models

arXiv:2609.15180v1 Announce Type: new Abstract: Vision-language models are increasingly explored for clinical prediction from electronic health records and medical images, where identifying unreliabl...

By Mingcheng Zhu, Jinning Liang, Tingting Zhu

llms multimodal

arXiv Machine Learning

Aug 21

Answer-Level Trust Selection for Physical Vision-Language Reasoning

arXiv:2608. 19807v1 Announce Type: new Abstract: Vision-language models (VLMs) can estimate physical quantities such as duration, speed, and acceleration from visual observations, but existing benchmarks primarily assess overall model performance against annotated ground truth.

By Rongyu Yu, Ke Niu, Fengxiang He

llms fine-tuning multimodal benchmarks

arXiv AI

Aug 11

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information

arXiv:2608. 09080v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks.

By Maryam Tahermazandarani, Adnan Mahmood, Fahmida Islam, Quan Z. Sheng

llms nlp

arXiv Computation and Language

Aug 21

Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models

arXiv:2606. 15910v2 Announce Type: replace Abstract: A vision-language model can answer a question about a chest radiograph or a pathology slide fluently and confidently while barely using the image, relying instead on language priors.

By Reza Khanmohammadi, Kundan Thind, Mohammad M. Ghassemi

llms agents multimodal

arXiv Machine Learning

Jul 14

The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Models

arXiv:2603. 16728v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly deployed in high-stakes settings where reliable uncertainty quantification (UQ) is as important as predictive accuracy.

By Robert Welch, Emir Konuk, Kevin Smith

llms multimodal

arXiv AI

Aug 14

How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures

arXiv:2608. 13267v1 Announce Type: cross Abstract: Existing vision-language model (VLM) benchmarks emphasize perception and reasoning accuracy (how well VLMs describe and reason about what they see in an image), with limited attention to behavioral reliability under uncertainty (how they behave when visual evidence is missing or misleading).

By Paul Osemudiame Oamen, Owusu-Banahene Osei, Ananya Mukherjee, Christian Greisinger, Steffen Eger, Pius Onobhayedo, Wei Zhao

llms multimodal benchmarks safety

One email a morning, machine-written

Related stories

Rethinking Correctness for Uncertainty Estimation in Clinical Prediction with Vision-Language Models

Answer-Level Trust Selection for Physical Vision-Language Reasoning

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information

Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models

The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Models

How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures