Modern large language models (LLMs) often exhibit overconfident errors, assigning high confidence to incorrect predictions and thereby undermining reliability in real-world use. Existing calibration m...
arXiv:2608. 13591v1 Announce Type: new Abstract: High-confidence errors in large language models are often treated as evidence of fragile internal inference.
arXiv:2608.22483v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is mis...
arXiv:2509.24988v2 Announce Type: replace-cross Abstract: Generating accurate and calibrated confidence estimates is critical for deploying LLMs in high-stakes or user-facing applications, and remain...
Large language models (LLMs) frequently produce confident yet factually incorrect responses when user inputs contain misleading premises, a phenomenon we attribute to fact perturbations in the input....
arXiv:2507. 02778v3 Announce Type: replace-cross Abstract: Although large language models (LLMs) have transformed AI, they still make errors and follow unproductive reasoning paths.
arXiv:2606. 09876v1 Announce Type: new Abstract: Large language models often express high confidence in answers that are wrong.
arXiv:2606. 11211v1 Announce Type: cross Abstract: The ability of large language models (LLMs) to express calibrated uncertainty is important for safe deployment.
arXiv:2604.08974v2 Announce Type: replace Abstract: Uncertainty quantification techniques measure confidence in language model outputs to support critical applications like hallucination detection an...
The paper investigates whether language models’ self-reported confidence is meaningful without additional training. By evaluating three training‑free signals—direct verbalization, post‑hoc probability estimates, and agreement across multiple generations—on 100 TriviaQA questions, the authors find that direct verbalization alone achieves high AUROC scores (0.956 and 0.937) for correctness prediction, while agreement-based methods perform noticeably worse. Re‑eliciting confidence for the same answers shows modest score shifts and occasional decision flips, and an audit of biography claims reveals only a small confidence gap between supported and contradicted statements.
The paper introduces DEDUCE, a three‑stage framework that turns large language models into proactive error correctors by detecting input fact errors, devising correction strategies, and delivering reliable answers. It also presents MisFactQA, a dataset of factual errors, and new metrics for robustness evaluation. Experiments on TruthfulQA, FalseQA, and MisFactQA show significant gains in accuracy and error correction across Qwen, LLaMA, and Gemma models.
arXiv:2607. 20526v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in settings where fluent but incorrect answers can be costly.