arXiv AI
Jul 7
The Anatomy of Uncertainty in LLMs
arXiv:2603. 24967v2 Announce Type: replace Abstract: Understanding why a large language model (LLM) is uncertain about the response is important for their reliable deployment.
Teaching models to express their uncertainty in words
Related stories
Hugging Face Trending Papers
Jun 22
The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models
Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.
arXiv AI
Jul 23
Rethinking Uncertainty Evaluation in Large Language Models
arXiv:2607. 19367v1 Announce Type: new Abstract: Calibration is the primary criterion for evaluating LLM confidence, but it is insufficient: it admits trivially incoherent estimators, depends on the evaluation distribution, and does not test the extent to which the estimation can be interpreted as a consistent, underlying probability function.
arXiv Machine Learning
Jun 11
What Uncertainties Do We Need for Dynamical Systems?
arXiv:2606. 11988v1 Announce Type: new Abstract: The distinction between aleatoric and epistemic uncertainty has received considerable attention in machine learning research, mainly in the context of supervised learning but also in other settings such as generative modeling.
arXiv Machine Learning
1d ago
Language models suffer from a curse of ambiguity
arXiv:2608. 15448v1 Announce Type: cross Abstract: Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever.
arXiv Machine Learning
Jun 19
Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence
arXiv:2606. 19353v1 Announce Type: cross Abstract: In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern: predictions are highly sensitive to both prompt design and the model's ability to understand the context, obscuring whether failures arise from data properties or model limitations.
arXiv AI
Jul 28
Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty
arXiv:2508. 08992v4 Announce Type: replace Abstract: Real-world decision-making often involves uncertainty expressed in linguistic rather than numerical terms, and Prospect Theory (PT) provides a classic framework for modeling human behavior under such uncertainty.
arXiv AI
Jun 9
From `May' to `Is': Certainty Distortion in Language Model Rewriting
arXiv:2606. 07951v1 Announce Type: cross Abstract: Humans increasingly turn to Language Models (LMs) in ways that shape beliefs and drive decisions, including discussing, rewriting, and summarizing information from scientific articles, news, and medical reports.
arXiv Machine Learning
Jun 30
Uncertainty-Aware Generation and Decision-Making Under Ambiguity
arXiv:2606. 30578v1 Announce Type: cross Abstract: With rapidly improving capabilities, Large Language Models (LLMs) are increasingly used in many complex real-world tasks.
arXiv AI
Jul 7
Failures and Successes to Learn a Core Conceptual Distinction from the Statistics of Language
arXiv:2607. 04523v1 Announce Type: cross Abstract: Generic statements like "tigers are striped" and "cars have radios" communicate information that is, in general, true.
arXiv AI
Aug 3
Benchmarking LLM Competence on Logical Inference over Probability Operators
arXiv:2607. 27405v2 Announce Type: replace-cross Abstract: Both expressions of uncertainty and inferences are ubiquitous in natural language, and valid inferences over natural-language expressions of uncertainty are necessary for not only everyday conversations but also for high-stakes domains such as medicine and law.
arXiv AI
Jun 3
Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models
arXiv:2606. 03846v1 Announce Type: cross Abstract: Large language models (LLMs) demonstrate remarkable performance across diverse tasks, but they often generate responses that appear plausible while being factually incorrect.