arXiv Machine Learning
Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jul 1
CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation
arXiv:2606. 32012v1 Announce Type: new Abstract: Uncertainty estimation has been a long-standing challenge in AI models; it amounts to "knowing what you don't know," and metacognition is notoriously difficult even for humans (cf.
arXiv AI
Jul 7
Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language
arXiv:2607. 03882v1 Announce Type: cross Abstract: LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate probabilistic information in natural language.
arXiv AI
Aug 11
MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks
arXiv:2507. 19634v4 Announce Type: replace-cross Abstract: Recent advances in large language models have laid the foundation for multimodal LLMs (MLLMs), which unify text, speech, and vision within a single framework.
arXiv Computation and Language
Aug 28
Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMs
The paper introduces Information-Guided Frontier Decoding (IGFD), a training‑free strategy for diffusion multimodal language models that prioritizes the commitment of reliable semantic tokens over fragile structural ones. IGFD ranks candidates by token confidence, neighborhood uncertainty, and structural commitment risk, and uses a dynamic frontier to limit selection to locally expandable regions. Across multiple multimodal benchmarks, IGFD consistently outperforms existing decoding methods while requiring no extra training or forward passes.
arXiv Machine Learning
Sep 21
Semantic Calibration Prevails Where Token Confidence Fails: Benchmarking Long-Form Scientific QA
The paper presents the first large‑scale benchmark for uncertainty quantification (UQ) calibration in long‑form scientific question answering, evaluating four UQ methods on 685,000 responses from up to 20 large language models across seven datasets. It shows that instruction tuning leads to token‑level probability polarization, undermining token‑level uncertainty signals, while reasoning model families differ in how they handle this effect. Only semantic consistency—consistency of the final answer—provides well‑calibrated outputs, demonstrating that semantic calibration remains robust in multi‑step, dependency‑rich reasoning.
arXiv AI
Jun 3
SeSE: Black-Box Uncertainty Quantification for Large Language Models Based on Structural Information Theory
arXiv:2511. 16275v4 Announce Type: replace-cross Abstract: Reliable uncertainty quantification (UQ) is essential for deploying large language models (LLMs) in safety-critical scenarios, as it enables them to abstain from responding when uncertain, thereby avoiding hallucinations, i.