arXiv AI October 19, 2026 By Ponhvoan Srey, Quang Minh Nguyen, Xiaobao Wu, Anh Tuan Luu

Towards Reliable Truth-Aligned Uncertainty Estimation in Large Language Models

The paper introduces Truth AnChoring (TAC), a lightweight post‑hoc calibration method that transforms raw uncertainty estimation scores into truth‑aligned correctness probabilities for large language models. It identifies that many existing UE metrics fail because they are derived from model behavior rather than factual correctness, leading to unstable performance across models and datasets. TAC improves calibration even with noisy, few‑shot supervision, offering a practical protocol for more reliable uncertainty estimation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 26

Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review

arXiv:2504.18346v4 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have been transformative across many domains. However, hallucination, i.e., confidently outputting incorrect inf...

By Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Leon Witt, Muhammad Asif Ali, Yukai Miao, Dan Li, Qingsong Wei

llms benchmarks safety

arXiv AI

Aug 24

Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models

arXiv:2608.21019v1 Announce Type: cross Abstract: Quantization is widely used to deploy large language models, but its effect on uncertainty behavior, such as confidence, margins, and abstention, is...

By Zhen Yang, Sizai Hou, Kaiwen Zheng, Yaofang Liu, Liang He, Yixuan Chen, Kangning Cui

llms nlp efficiency benchmarks

arXiv Computation and Language

Aug 25

Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models

arXiv:2608.22483v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is mis...

By Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Li Chen

llms fine-tuning safety

Hugging Face Trending Papers

Jun 22

The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models

Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.

llms benchmarks

arXiv Machine Learning

Jun 2

How to Correctly Report LLM-as-a-Judge Evaluations

arXiv:2511. 21140v4 Announce Type: replace Abstract: Large language models (LLMs) are widely used as scalable evaluators of model responses in lieu of human annotators.

By Chungpa Lee, Thomas Zeng, Jongwon Jeong, Jy-yong Sohn, Kangwook Lee

llms safety

arXiv Computation and Language

Sep 15

Can We Trust the Judges? Validation of Factuality Evaluation Methods via Answer Perturbation

arXiv:2609.15561v1 Announce Type: new Abstract: Evaluating the factual correctness of large language models (LLMs) is vital for many applications. But are our evaluation tools themselves trustworthy?...

By Sarra Gharsallah, Adele Robaldo, Mariia Tokareva, Giovanni Gatti Pinheiro, Ilyana Guendouz, Rapha\"el Troncy, Paolo Papotti, Pietro Michiardi

llms