arXiv AI By Dharshan Kumaran

Reported Confidence in LLMs Tracks Commitment More Than Correctness

Read the original on arXiv AI →

arXiv:2606. 29490v1 Announce Type: cross Abstract: Confidence is an estimate of the probability that a chosen answer is correct.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Jul 14

The Computational Basis of Confidence in Large Language Models

Reliable confidence -- the probability that a model's own answer is correct -- is essential for the trustworthy deployment of language models. Existing work has largely evaluated confidence by how well it predicts correctness and whether it is calibrated, leaving open a more fundamental question: what does the confidence signal itself represent?

arXiv AI
Sep 21

How do LLMs Compute Verbal Confidence

arXiv:2603.17839v4 Announce Type: replace-cross Abstract: Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from...

By Dharshan Kumaran, Arthur Conmy, Federico Barbero, Simon Osindero, Viorica Patraucean, Petar Veli\v{c}kovi\'c
Hugging Face Trending Papers
Aug 27

Calibrated Enough to Know, Not Calibrated to Act: Fabricated Evidence Makes LLM Agents Commit to the Unknowable

The paper shows that large language model agents are far more likely to commit to a directional answer when presented with a professional-looking market panel, even if the panel’s data is fabricated. Across 12 frontier models, commitment rates jump from 6.5 % for a bare question to 54.0 % with evidence, and remain high (≈37 %) even when all numbers are invented. The study finds that the act/don’t‑act decision gate is the key failure point, can be trained to reduce false commitments, but is fragile to response format changes.