arXiv Machine Learning June 19, 2026 By Jinseok Chung, Minkyoung Song, Hyunji Jung, Namhoon Lee

Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence

Read the original on arXiv Machine Learning →

arXiv:2606. 19353v1 Announce Type: cross Abstract: In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern: predictions are highly sensitive to both prompt design and the model's ability to understand the context, obscuring whether failures arise from data properties or model limitations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 29

On the Effect of Uncertainty on Layer-wise Inference Dynamics

arXiv:2507. 06722v2 Announce Type: replace-cross Abstract: Understanding how large language models (LLMs) internally represent and process their predictions is central to detecting uncertainty and preventing hallucinations.

By Sunwoo Kim, Haneul Yoo, Alice Oh

llms safety

arXiv AI

Sep 15

Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns

arXiv:2509.24988v2 Announce Type: replace-cross Abstract: Generating accurate and calibrated confidence estimates is critical for deploying LLMs in high-stakes or user-facing applications, and remain...

By Hanqi Xiao, Vaidehi Patil, Hyunji Lee, Elias Stengel-Eskin, Mohit Bansal

llms benchmarks

arXiv Machine Learning

Aug 7

A Unified Risk View of Uncertainty: Posterior Risk for Disentanglement and Evaluation Beyond Proxies

arXiv:2608. 05995v1 Announce Type: new Abstract: Reliable uncertainty estimates are critical in safety-sensitive applications, where understanding the sources of predictive uncertainty is essential.

By Frieder Wizgall, Georg Tirpitz, Moritz Seiler, Kerstin Ritter, B\'alint Mucs\'anyi

benchmarks safety

arXiv AI

Jul 7

The Anatomy of Uncertainty in LLMs

arXiv:2603. 24967v2 Announce Type: replace Abstract: Understanding why a large language model (LLM) is uncertain about the response is important for their reliable deployment.

By Aditya Taparia, Ransalu Senanayake, Kowshik Thopalli, Vivek Narayanaswamy

llms safety

arXiv AI

Aug 5

LogitScope: A Framework for Analyzing LLM Uncertainty Through Information Metrics

arXiv:2603. 24929v2 Announce Type: replace Abstract: Understanding and quantifying uncertainty in large language model (LLM) outputs is critical for reliable deployment.

By Farhan Ahmed, Yuya Jeremy Ong, Chad DeLuca

llms safety

arXiv AI

Jun 3

Quantifying Faithful Confidence Expression in Large Reasoning Models

arXiv:2606. 03969v1 Announce Type: cross Abstract: Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)--the alignment between models' intrinsic and (linguistically) expressed confidence--is a persistent failure mode.

By Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu, Arman Cohan

llms safety

One email a morning, machine-written

Related stories

On the Effect of Uncertainty on Layer-wise Inference Dynamics

Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns

A Unified Risk View of Uncertainty: Posterior Risk for Disentanglement and Evaluation Beyond Proxies

The Anatomy of Uncertainty in LLMs

LogitScope: A Framework for Analyzing LLM Uncertainty Through Information Metrics

Quantifying Faithful Confidence Expression in Large Reasoning Models