arXiv AI September 17, 2026 By Jisoo Yang, Jaeho Han, Trung X. Pham, Junyeong Kim

The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models

The paper investigates how Vision‑Language Models (VLMs) often report high confidence even after self‑correcting or arriving at wrong answers, a phenomenon the authors attribute to the verbalized confidence being largely independent of the model’s reasoning trajectory. By analyzing content variation, token masking, and hesitation markers, the authors demonstrate that confidence does not adequately reflect the actual reasoning process and that calibration training can sometimes worsen this disconnect. To address this blind spot, they introduce the Trajectory‑Grounding Score (TGS) in two forms—TGS‑self and TGS‑pair—and propose TGS‑Bench, a suite of 10 benchmarks that reveal divergences between conventional calibration metrics and trajectory‑grounded confidence.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 14

REHEARSE: Experiential Rehearsal for Verbal Confidence Calibration in Large Language Models

arXiv:2508. 14390v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often express verbal confidence that is poorly aligned with actual correctness, limiting their reliability in safety-critical applications.

By Ke Fang, Tianyi Zhao, Qianwen Wang, Lu Cheng

llms benchmarks safety

arXiv AI

Aug 3

Step-Level Visual Grounding Faithfulness Predicts Out-of-Distribution Generalization in Long-Horizon Vision-Language Models

arXiv:2603. 06828v2 Announce Type: replace-cross Abstract: We uncover a behavioral law of long-horizon vision-language models: models that maintain temporally grounded beliefs generalize better.

By Md Ashikur Rahman, Md Arifur Rahman, Niamul Hassan Samin, Abdullah Ibne Hanif Arean, Juena Ahmed Noshin

llms multimodal benchmarks

arXiv AI

Jul 7

Iterative Visual Thinking and the Self-Correction Mirage in VLM Grounding

arXiv:2606. 13156v2 Announce Type: replace-cross Abstract: Letting a vision-language model (VLM) think longer at test time has driven much recent progress.

By Animesh Tripathy, Aswanth Krishnan

llms multimodal

arXiv Computer Vision

2d ago

SAVOR: Self-Aware Visual Grounding via Confidence-Calibrated Reinforcement Learning for Multimodal Hallucination Mitigation

SAVOR is a training framework for multimodal large language models that adds token and answer confidence to the output schema, optimises a Group Relative Policy Optimisation objective to penalise calibration error and poor abstention, and uses the learned confidence at inference to revisit visual evidence only when uncertain. Experiments on POPE, HallusionBench, AMBER, and MMHal-Bench with InternVL3-8B and Qwen3-VL-8B backbones show that SAVOR reduces hallucination while maintaining general capability on MME and MMBench, achieving lower Expected Calibration Error than DPO and decoding baselines.

By Zixiu Ding, Zilin Zhao, Yingjie He, Xinlang Kang, Guansu Wang, Wei Zhang

llms nlp reinforcement-learning multimodal safety

arXiv Machine Learning

Sep 10

Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Models

arXiv:2609.06419v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) require confidence that reflects both answer correctness and patient-specific visual evidence. Recent GRPO-base...

By Yangyang Xie, Ke Hao, Jiaqi Liu, Yun Gu, Xinglin Zhang

llms multimodal benchmarks

arXiv AI

Aug 6

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models

arXiv:2604. 14888v3 Announce Type: replace-cross Abstract: Recent advances in vision language models (VLMs) offer reasoning capabilities, yet how these unfold and integrate visual and textual information remains unclear.

By Danae S\'anchez Villegas, Samuel Lewis-Lim, Nikolaos Aletras, Desmond Elliott

llms multimodal safety

One email a morning, machine-written

Related stories

REHEARSE: Experiential Rehearsal for Verbal Confidence Calibration in Large Language Models

Step-Level Visual Grounding Faithfulness Predicts Out-of-Distribution Generalization in Long-Horizon Vision-Language Models

Iterative Visual Thinking and the Self-Correction Mirage in VLM Grounding

SAVOR: Self-Aware Visual Grounding via Confidence-Calibrated Reinforcement Learning for Multimodal Hallucination Mitigation

Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Models

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models