arXiv AI
Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Jun 22
The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models
Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.
arXiv AI
Jul 23
Rethinking Uncertainty Evaluation in Large Language Models
arXiv:2607. 19367v1 Announce Type: new Abstract: Calibration is the primary criterion for evaluating LLM confidence, but it is insufficient: it admits trivially incoherent estimators, depends on the evaluation distribution, and does not test the extent to which the estimation can be interpreted as a consistent, underlying probability function.
arXiv AI
Jun 19
A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Models
arXiv:2606. 19868v1 Announce Type: new Abstract: Although large language models (LLMs) have shown strong capabilities across a wide range of tasks, their outputs often remain unreliable and may contain hallucinations, making uncertainty estimation (UE) essential for building trustworthy LLMs.
arXiv AI
Aug 21
Improved Confidence Estimates for Black-Box Large Language Models
arXiv:2608. 19323v1 Announce Type: cross Abstract: Uncertainty quantification (UQ) is essential for the safe deployment of large language models (LLMs).
arXiv AI
Jul 24
ConfidenceBench: Evaluating Confidence Calibration in Large Language Models
arXiv:2607. 20526v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in settings where fluent but incorrect answers can be costly.
arXiv Machine Learning
Jun 2
How to Correctly Report LLM-as-a-Judge Evaluations
arXiv:2511. 21140v4 Announce Type: replace Abstract: Large language models (LLMs) are widely used as scalable evaluators of model responses in lieu of human annotators.