arXiv AI

ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

arXiv:2607. 20526v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in settings where fluent but incorrect answers can be costly.

arXiv AI
Jul 23

Rethinking Uncertainty Evaluation in Large Language Models

arXiv:2607. 19367v1 Announce Type: new Abstract: Calibration is the primary criterion for evaluating LLM confidence, but it is insufficient: it admits trivially incoherent estimators, depends on the evaluation distribution, and does not test the extent to which the estimation can be interpreted as a consistent, underlying probability function.

By Krish Matta, Atharv Naphade, Andy Zou