arXiv AI

Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

arXiv:2608. 13591v1 Announce Type: new Abstract: High-confidence errors in large language models are often treated as evidence of fragile internal inference.

arXiv Computation and Language

Sep 18

An Analysis of Training-Free Self-Reported Confidence in Language Models

The paper investigates whether language models’ self-reported confidence is meaningful without additional training. By evaluating three training‑free signals—direct verbalization, post‑hoc probability estimates, and agreement across multiple generations—on 100 TriviaQA questions, the authors find that direct verbalization alone achieves high AUROC scores (0.956 and 0.937) for correctness prediction, while agreement-based methods perform noticeably worse. Re‑eliciting confidence for the same answers shows modest score shifts and occasional decision flips, and an audit of biography claims reveals only a small confidence gap between supported and contradicted statements.

By Lukas Meyer, Sofia Rossi, Wei Chen, Thomas Laurent, Yiming Li

llms benchmarks

arXiv Machine Learning

Jun 5

CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction

arXiv:2606. 05799v1 Announce Type: new Abstract: Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's {\em behavioral robustness} to irrelevant or misleading information.

By Mohammad Anas Jawad, Cornelia Caragea

llms benchmarks

arXiv Computation and Language

Sep 22

Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning

arXiv:2604.08974v2 Announce Type: replace Abstract: Uncertainty quantification techniques measure confidence in language model outputs to support critical applications like hallucination detection an...

By Lorenzo Jaime Yu Flores, Cesare Spinoso di-Piano, Jackie Chi Kit Cheung

llms nlp fine-tuning safety

arXiv Computation and Language

Aug 25

Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models

arXiv:2608.22483v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is mis...

By Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Li Chen

llms fine-tuning safety

arXiv AI

Jul 24

ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

arXiv:2607. 20526v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in settings where fluent but incorrect answers can be costly.

By Matthew ffrench-Constant, Daniel Yang, Xinmeng Huang, Sanyam Kapoor

llms benchmarks

arXiv AI

Sep 7

When Do Internal Probes Beat Reading the Answer? Miscalibrated Readouts and Behavior-Concealed Knowledge in Language Models

A 0.6B language model consistently answers YES to 1,200 logical tests, yet its behavior shows no discrimination. Linear probes reveal the correct verdict with high AUC (0.96) and transfer to unseen structures, but a single scalar readout fails due to a saturated decision threshold offset by +4.6 σ. Adjusting this threshold restores behavior accuracy from 50 % to 81 % and improves higher‑scale models, demonstrating that miscalibrated readouts, not hidden knowledge loss, drive performance gaps.

By Gnaneswar Villuri, Hashmath Shaik, Alex Doboli

llms

arXiv AI

Aug 6

Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models

arXiv:2608. 05064v1 Announce Type: cross Abstract: Small open-weight language models increasingly run in private, offline, and cost-sensitive settings, where the key deployment question is not only what a model answers but when it should defer to a human.

By Jianru Shen

llms

arXiv Machine Learning

Jun 10

Calibrating Overconfidence Without Sacrificing Confidence: Probe-Conditioned Head Intervention for LLMs

arXiv:2606. 09876v1 Announce Type: new Abstract: Large language models often express high confidence in answers that are wrong.

By Ke Li, Chongzhe Zhang, Zifan Zeng, Feng Liu, Qunli Zhang, Zheng Hu

llms

arXiv AI

Sep 25

PROOF: Profiling Reliability of Object-Level Facts in Large Language Models

PROOF is a benchmark that profiles the reliability of object-level facts in instruction-tuned language models by converting a frozen Wikidata snapshot into 18,486 English multiple-choice questions grounded in 11,779 semantic facts across 101 classes, 392 properties, and 14 domains. Each question includes an explicit "I don't know" option, a "No correct option" control, and nine controlled formulations, with 1,849 questions designed as no-correct-option traps. The study evaluates 18 open-weight model deployments on 166,374 prompts, revealing wide variability in factual accuracy, sensitivity to wording changes, and the impact of decoder perturbations.

By Andrei Chetvergov, Mikhail Solovev, Timofei Sivoraksha, Stepan Ukolov, Valeriia Kuschenko, Alexander Evseev, Sergey Bolovtsov

llms benchmarks safety

arXiv Machine Learning

2d ago

Also Small Models Can Reasonably Self-Evaluate Their Confidence

arXiv:2609.39478v1 Announce Type: new Abstract: This study systematically evaluates self-evaluation-based uncertainty quantification across different language models of varying sizes on question-answ...

By Idil Kapikiran, Thomas Decker, Thomas Runkler

llms

arXiv AI

Jul 17

Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect

arXiv:2607. 14111v1 Announce Type: cross Abstract: Can small language models detect and report on perturbations their own internal activations?

By Ely Hahami, Ishaan Sinha, Lavik Jain

llms fine-tuning benchmarks safety

arXiv Computation and Language

Aug 31

When Linguistic and Internal Confidence Diverge in Large Language Models

arXiv:2608.28382v1 Announce Type: new Abstract: Users often ask large language models (LLMs) to report how confident they are, but it is unclear whether such linguistic confidence tracks the model's...

By Hefan Zhang, Bingquan Zhang, Ming Cheng, Saeed Hassanpour, Weicheng Ma, Soroush Vosoughi

llms safety

Related stories