arXiv AI August 20, 2026 By Davide Mazzaccara, Leonardo Bertolazzi, Raffaella Bernardi

Different Facets of Verbalised Overconfidence: an Interpretability Study

The study examines how the large language model Qwen3-4B exhibits overconfidence across three expressions of uncertainty—verbal epistemic markers, abstention, and numeric confidence scores—using controlled reasoning scenarios that vary logical necessity and possibility. Findings confirm a pronounced overconfidence tendency, especially when numeric scores are requested, and reveal that certainty is generated by a broad coalition of shared features while uncertainty relies on a sparse set of dedicated features. By intervening on these uncertainty features, the researchers demonstrate causal evidence of the imbalance driving overconfidence and show that this intervention can mitigate overconfident errors across different settings, languages, and an out‑of‑distribution modality task.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

3d ago

Measuring Mechanistic Uncertainty in Large Language Models via Causal Robustness

The paper introduces mechanistic uncertainty, a measure of how stable a large language model’s internal mechanisms are when producing a response. It defines the causal robustness margin as the smallest internal intervention needed to change the output, and implements this using Jacobian Lens steering vectors. Experiments on three open‑source models and five QA datasets show that incorrect or uncertain answers have smaller margins, while correctly learned facts exhibit larger margins after training, indicating that causal robustness complements traditional output‑based uncertainty methods.

By Aneesh Komanduri, Majid Afshar, Timothy A. Miller, Yanjun Gao

llms safety

arXiv AI

Aug 11

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information

arXiv:2608. 09080v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks.

By Maryam Tahermazandarani, Adnan Mahmood, Fahmida Islam, Quan Z. Sheng

llms nlp

arXiv AI

Jun 2

The Role of Ambiguity in Error Prediction via Uncertainty Quantification

arXiv:2606. 02093v1 Announce Type: cross Abstract: The task of Error Prediction, namely predicting whether a model output is correct, is commonly tackled with Uncertainty Quantification (UQ).

By Ieva Raminta Stali\=unait\.e, James Bishop, Andreas Vlachos

llms nlp

arXiv AI

Aug 17

Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

arXiv:2608. 13591v1 Announce Type: new Abstract: High-confidence errors in large language models are often treated as evidence of fragile internal inference.

By Akira Okutomi

llms

arXiv Machine Learning

Jun 29

On the Effect of Uncertainty on Layer-wise Inference Dynamics

arXiv:2507. 06722v2 Announce Type: replace-cross Abstract: Understanding how large language models (LLMs) internally represent and process their predictions is central to detecting uncertainty and preventing hallucinations.

By Sunwoo Kim, Haneul Yoo, Alice Oh

llms safety

arXiv AI

Jul 7

Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

arXiv:2607. 03882v1 Announce Type: cross Abstract: LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate probabilistic information in natural language.

By Diego Cerda-Mardini, Sarath Chandar, Sreenath Madathil

llms

One email a morning, machine-written

Related stories

Measuring Mechanistic Uncertainty in Large Language Models via Causal Robustness

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information

The Role of Ambiguity in Error Prediction via Uncertainty Quantification

Stable Miscalibration in Large Language Models: A Practical View of High-Confidence Errors

On the Effect of Uncertainty on Layer-wise Inference Dynamics

Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language