arXiv AI
Beyond Calibration: Temporal LLM Confidence Assessment Across Languages and Local Settings
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Oct 20
Beyond Calibration: Temporal LLM Confidence Assessment Across Languages and Local Settings
Large language models are expected to estimate their own uncertainty, with confidence signals used to guide retrieval, adaptive compute allocation, routing, and selective abstention. We investigate wh...
arXiv AI
Jul 9
Future Confidence Distillation in Large Language Models
arXiv:2607. 07626v1 Announce Type: cross Abstract: Reliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability.
arXiv AI
Sep 21
How do LLMs Compute Verbal Confidence
arXiv:2603.17839v4 Announce Type: replace-cross Abstract: Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from...
arXiv Computation and Language
Sep 30
LLMs learn different forms of metacognition when trained to predict their own accuracy
The study trains ten open‑weight large language models (LLMs) to predict their own accuracy on factual multiple‑choice questions before answering. Results show that the models’ confidence signals split into two distinct patterns: early in training, confidence aligns with output consistency (how concentrated the answer distribution is), while later, it aligns with true accuracy but only on data similar to the training set. This indicates that calibration training may not universally teach LLMs to detect their own errors.
arXiv Computation and Language
Sep 22
Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning
arXiv:2604.08974v2 Announce Type: replace Abstract: Uncertainty quantification techniques measure confidence in language model outputs to support critical applications like hallucination detection an...
arXiv AI
Aug 14
REHEARSE: Experiential Rehearsal for Verbal Confidence Calibration in Large Language Models
arXiv:2508. 14390v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often express verbal confidence that is poorly aligned with actual correctness, limiting their reliability in safety-critical applications.