← Back to all news
arXiv AI By Qinhong Lin, Yinglun Feng, Yuhao Zhang, Zhongliang Yang, Linna Zhou

Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation

Read the original on arXiv AI →

The paper introduces Label-Confidence-Aware Uncertainty Quantification (LCA-UQ), a method that uses Pointwise Kullback-Leibler divergence to align global entropy from multiple stochastic samples with the local confidence of a candidate answer. By bridging this gap, LCA-UQ improves the reliability and stability of uncertainty assessments in natural language generation. Experiments on popular LLMs and NLP datasets show that label sources significantly influence classification and that LCA-UQ outperforms existing uncertainty estimation approaches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

Hugging Face Trending Papers
Jun 22

The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models

Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.

llmsbenchmarks
More like this →
arXiv Machine Learning
Aug 27

Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantification

The paper introduces Functional Entropy, a new uncertainty quantification technique for assessing the functional correctness of code generated by large language models. It evaluates token‑probability and sampling‑based methods across three programming languages and five LLMs, finding that token‑probability approaches generalize well while NLI‑based sampling fails due to semantic clustering. Functional equivalence methods, which replace NLI with an LLM‑based functional assessment, achieve superior AUROC and calibration in most model‑benchmark combinations.

By Dylan Bouchard, Mohit Singh Chauhan, Zeya Ahmad, Ho-Kyeong Ra
llmsbenchmarkssafety
More like this →