arXiv Machine Learning June 19, 2026 By Jinseok Chung, Minkyoung Song, Hyunji Jung, Namhoon Lee

Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence

Read the original on arXiv Machine Learning →

arXiv:2606. 19353v1 Announce Type: cross Abstract: In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern: predictions are highly sensitive to both prompt design and the model's ability to understand the context, obscuring whether failures arise from data properties or model limitations.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 29

On the Effect of Uncertainty on Layer-wise Inference Dynamics

arXiv:2507. 06722v2 Announce Type: replace-cross Abstract: Understanding how large language models (LLMs) internally represent and process their predictions is central to detecting uncertainty and preventing hallucinations.

By Sunwoo Kim, Haneul Yoo, Alice Oh

llms safety

arXiv Machine Learning

Aug 7

A Unified Risk View of Uncertainty: Posterior Risk for Disentanglement and Evaluation Beyond Proxies

arXiv:2608. 05995v1 Announce Type: new Abstract: Reliable uncertainty estimates are critical in safety-sensitive applications, where understanding the sources of predictive uncertainty is essential.

By Frieder Wizgall, Georg Tirpitz, Moritz Seiler, Kerstin Ritter, B\'alint Mucs\'anyi

benchmarks safety

arXiv AI

Jul 7

The Anatomy of Uncertainty in LLMs

arXiv:2603. 24967v2 Announce Type: replace Abstract: Understanding why a large language model (LLM) is uncertain about the response is important for their reliable deployment.

By Aditya Taparia, Ransalu Senanayake, Kowshik Thopalli, Vivek Narayanaswamy

llms safety

arXiv AI

Aug 5

LogitScope: A Framework for Analyzing LLM Uncertainty Through Information Metrics

arXiv:2603. 24929v2 Announce Type: replace Abstract: Understanding and quantifying uncertainty in large language model (LLM) outputs is critical for reliable deployment.

By Farhan Ahmed, Yuya Jeremy Ong, Chad DeLuca

llms safety

arXiv AI

Jun 3

Quantifying Faithful Confidence Expression in Large Reasoning Models

arXiv:2606. 03969v1 Announce Type: cross Abstract: Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)--the alignment between models' intrinsic and (linguistically) expressed confidence--is a persistent failure mode.

By Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu, Arman Cohan

llms safety

Hugging Face Trending Papers

Jun 22

The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models

Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.

llms benchmarks

Related stories

On the Effect of Uncertainty on Layer-wise Inference Dynamics

A Unified Risk View of Uncertainty: Posterior Risk for Disentanglement and Evaluation Beyond Proxies

The Anatomy of Uncertainty in LLMs

LogitScope: A Framework for Analyzing LLM Uncertainty Through Information Metrics

Quantifying Faithful Confidence Expression in Large Reasoning Models

The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models