arXiv Machine Learning

Semantic Calibration Prevails Where Token Confidence Fails: Benchmarking Long-Form Scientific QA

The paper presents the first large‑scale benchmark for uncertainty quantification (UQ) calibration in long‑form scientific question answering, evaluating four UQ methods on 685,000 responses from up to 20 large language models across seven datasets. It shows that instruction tuning leads to token‑level probability polarization, undermining token‑level uncertainty signals, while reasoning model families differ in how they handle this effect. Only semantic consistency—consistency of the final answer—provides well‑calibrated outputs, demonstrating that semantic calibration remains robust in multi‑step, dependency‑rich reasoning.

arXiv AI

Sep 30

Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs

The paper introduces Divergent Token Confidence (DTC), a method that estimates large language model confidence by counting tokens where two models strongly disagree during decoding. DTC uses Jensen-Shannon divergence between next-token distributions along the same reasoning trajectory and shows a near-negative correlation with answer accuracy. Experiments on multiple model families and six mathematical benchmarks demonstrate that DTC improves calibration over traditional probability-based and verbalized baselines, achieving lower expected calibration errors in both white-box and black-box settings.

By Feiyang Li, Shengjing Liu, Qi Zhan, Sijie Cheng, Weiqing Wang, Hongwen Chen, Yuxuan Yang, Wen Wang, Yile Wang, Hui Huang

llms benchmarks

Hugging Face Trending Papers

Sep 29

Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs

As the chain-of-thought reasoning capabilities of large language models improve, evaluating and calibrating their reasoning confidence is becoming increasingly important for quantifying the uncertaint...

llms benchmarks

arXiv Computation and Language

Sep 22

Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning

arXiv:2604.08974v2 Announce Type: replace Abstract: Uncertainty quantification techniques measure confidence in language model outputs to support critical applications like hallucination detection an...

By Lorenzo Jaime Yu Flores, Cesare Spinoso di-Piano, Jackie Chi Kit Cheung

llms nlp fine-tuning safety

arXiv Machine Learning

Aug 11

From token probabilities to calibrated confidence: An empirical study of mathematical question answering

arXiv:2608. 07827v1 Announce Type: new Abstract: Confidence estimation for large language models (LLMs) aims to estimate the probability that a generated answer is correct, while calibration aligns these estimates with empirical accuracy.

By Avery Ma, Lorne Schell, Vin Bhaskara, Leila Pishdad

llms nlp

Hugging Face Trending Papers

Oct 6

U-Space: Uncovering When and Why Uncertainty Arises in Language Models

Large language models are informing decisions with ever-higher stakes. As the consequences of their errors grow, a central question becomes harder to ignore: how much can we trust an individual answer...

llms benchmarks safety

arXiv AI

Jul 8

Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs

arXiv:2607. 06327v1 Announce Type: cross Abstract: Uncertainty estimation (UE) enables LLM-powered systems to recognize when to abstain, yet existing research has predominantly focused on English.

By Andrea Alfarano, Andrea Bacciu, Saab Mansour, Amin Mantrach, Marcello Federico

llms rag

arXiv AI

Jun 10

Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models

arXiv:2606. 09856v1 Announce Type: cross Abstract: Post-training Large Language Models (LLMs) for reasoning typically focuses on deductive tasks such as mathematics and coding where correctness is verifiable.

By Liyi Zhang, Akshay K. Jagadish, Brenden M. Lake, Thomas L. Griffiths

llms agents fine-tuning benchmarks safety

Hugging Face Trending Papers

Jul 7

SpanUQ: Span-Level Uncertainty Quantification for Large Language Model Generation

Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self-refinement in LLM generation. However, existing approaches operate at suboptimal granularities: token-level scores lack semantic coherence, while sequence-level scores fail to localize errors.

llms benchmarks

arXiv AI

Jun 3

Quantifying Faithful Confidence Expression in Large Reasoning Models

arXiv:2606. 03969v1 Announce Type: cross Abstract: Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)--the alignment between models' intrinsic and (linguistically) expressed confidence--is a persistent failure mode.

By Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu, Arman Cohan

llms safety

arXiv Computation and Language

Sep 25

Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations

The paper introduces a new evaluation framework for confidence estimation in large language models, focusing on three properties: robustness to prompt changes, stability across semantically equivalent answers, and sensitivity to semantically different answers. It demonstrates that existing confidence estimation methods perform well on robustness and stability but often fail to detect differences in answer meaning, revealing gaps in current evaluation practices. The framework aims to guide the selection of confidence estimators for practical applications.

By Yuxi Xia, Dennis Ulmer, Terra Blevins, Yihong Liu, Hinrich Sch\"utze, Benjamin Roth

llms safety

arXiv AI

Oct 5

Test-time Calibration Learning for Large Language Model Reasoning

The paper introduces Test-Time Calibration Learning (TTCL), a label‑free framework that adapts both reasoning accuracy and verbalized confidence of large language models directly on unlabeled target‑task data. TTCL generates self‑supervision signals from multiple model responses, enabling calibration without ground‑truth labels and proving theoretically as a bounded surrogate for the ideal calibration objective. Experiments on mathematical reasoning and factual question answering show consistent improvements, with base models gaining an average 40.13% in accuracy and 70.80% in ECE reduction across eight benchmarks, and further gains under domain shift.

By Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Shixiong Kai, Mingxuan Yuan, Bo Han

llms nlp reinforcement-learning benchmarks

arXiv AI

Aug 12

TrAC: Trace-Conditioned Answer Consistency for Efficient Uncertainty Quantification in LLMs

arXiv:2608. 00422v2 Announce Type: replace Abstract: Large language models (LLMs) can generate fluent reasoning traces that nevertheless lead to incorrect answers, making response-level uncertainty estimation important for abstention, human review, and adaptive compute allocation.

By Dahai Yu, Lin Jiang, Rongchao Xu, Guang Wang

llms benchmarks

Related stories