arXiv Machine Learning

Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation

arXiv:2607. 22034v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly deployed on consumer hardware where input images are degraded by compression, camera shake, and poor lighting.

arXiv Machine Learning

Jul 28

Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation

arXiv:2607. 24440v1 Announce Type: cross Abstract: Vision-language models (VLMs) deployed on consumer hardware must decide when to answer and when to defer, and that decision depends on having a confidence signal that tracks correctness.

By M M Asif Ferdous

llms efficiency multimodal

arXiv AI

5d ago

How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures

arXiv:2608. 13267v1 Announce Type: cross Abstract: Existing vision-language model (VLM) benchmarks emphasize perception and reasoning accuracy (how well VLMs describe and reason about what they see in an image), with limited attention to behavioral reliability under uncertainty (how they behave when visual evidence is missing or misleading).

By Paul Osemudiame Oamen, Owusu-Banahene Osei, Ananya Mukherjee, Christian Greisinger, Steffen Eger, Pius Onobhayedo, Wei Zhao

llms multimodal benchmarks safety

arXiv Machine Learning

Aug 12

Predictive Entropy as a Joint Screen for Error and Paraphrase Instability in Medical Vision-Language Models

arXiv:2604. 08941v2 Announce Type: replace Abstract: Medical Vision-Language Models (VLMs) answering binary presence questions on chest radiographs can fail in two linked ways: they are confidently wrong, and they change answers when a clinically equivalent question is rephrased.

By Binesh Sadanandan, Vahid Behzadan

llms multimodal

arXiv AI

Jun 30

Early Warning Signals for OpenVLA Failure under Visual Distribution Shift

arXiv:2606. 29699v1 Announce Type: cross Abstract: Vision Language Action models combine perception, language grounding, and control in a single policy, but their failures are hard to diagnose once visual conditions shift.

By Dipesh Tharu Mahato, Rachel Ren

robotics benchmarks

arXiv AI

Jul 10

When Thinking Hurts: Epistemic Signals in the Reasoning Chains of Visual Language Models

arXiv:2607. 08059v1 Announce Type: cross Abstract: Uncertainty quantification for visual language models (VLMs) conventionally targets the answer token distribution.

By Mayank Singal

llms safety

arXiv AI

Jun 17

Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models

arXiv:2606. 17389v1 Announce Type: cross Abstract: Multimodal Foundation Models are increasingly used as reasoning agents, making reliability, knowing when a model may hallucinate, critical.

By Logan Mann, Yi Xia, Ajit Saravanan, Ishan Dave, Saadullah Ismail, Shikhar Shiromani, Emily Huang, Ruizhe Li, Kevin Zhu

llms agents multimodal

arXiv AI

Jul 3

Assessing VLM Reliability for Medical Image Quality Evaluation Under Corruption and Bias

arXiv:2607. 01973v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly applied in medical tasks such as pathology description, report generation, and visual question answering.

By Sofiane Ouaari, Kevin Vorwalder, Nico Pfeifer

llms rag nlp multimodal benchmarks safety

arXiv AI

Aug 3

Step-Level Visual Grounding Faithfulness Predicts Out-of-Distribution Generalization in Long-Horizon Vision-Language Models

arXiv:2603. 06828v2 Announce Type: replace-cross Abstract: We uncover a behavioral law of long-horizon vision-language models: models that maintain temporally grounded beliefs generalize better.

By Md Ashikur Rahman, Md Arifur Rahman, Niamul Hassan Samin, Abdullah Ibne Hanif Arean, Juena Ahmed Noshin

llms multimodal benchmarks

arXiv AI

5d ago

TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint

arXiv:2608. 13167v1 Announce Type: cross Abstract: When visual evidence is occluded or chaotic, models should abstain.

By Fnu Pramono, John Cai, Sourabh Kulkarni

llms multimodal benchmarks

arXiv Machine Learning

Jun 26

Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA

arXiv:2606. 27023v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) applied to Medical Visual Question Answering (VQA) tend to produce overconfident outputs regardless of actual correctness, and existing verbalized confidence calibration methods, developed primarily for text only LLMs, do not account for the multimodal nature of medical image understanding.

By Eren Senoglu, Federico Toschi, Nicolo Brunello, Andrea Sassella, Mark James Carman

llms nlp fine-tuning multimodal benchmarks safety

arXiv AI

Jul 22

When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models

arXiv:2603. 03989v2 Announce Type: replace-cross Abstract: When visual evidence is ambiguous, vision models must decide how to interpret face-like patterns.

By Qianpu Chen, Derya Soydaner, Rob Saunders

llms computer-vision multimodal safety

arXiv AI

Jun 2

Asking Is Not Enough: Protocol Sensitivity in LLM Confidence Calibration

arXiv:2605. 27752v2 Announce Type: replace Abstract: LLM confidence calibration is often evaluated by comparing two signals: token-probability scores and verbalized confidence.

By Hankyeol Kim, Pilsung Kang

llms benchmarks

Related stories