← Back to all news
arXiv AI By Bhushan Kashinath Joshi

Confident at the moment of action: belief miscalibration in LLM play under hidden information

Read the original on arXiv AI →

The paper investigates whether large language models (LLMs) correctly gauge their confidence when acting in a hidden‑information chess variant. In experiments where the location of a hidden royal piece is repeatedly relocated, the models’ stated probabilities about the piece’s position were almost never accurate at high confidence levels, with a calibration deficit concentrated in those high‑confidence events. Across multiple model configurations and providers, the same pattern emerged, and conventional evaluation metrics such as legality, cost, latency, and completion rate were found to be uncorrelated with belief quality, yet a model could still win the game despite poor confidence estimates.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 21

Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions

The paper investigates why large language models (LLMs) struggle in strategic decision-making under incomplete information. It identifies two key gaps: an observation‑belief gap where LLMs’ internal representations of game states are accurate but brittle, and a belief‑action gap where converting these internal beliefs into actions is weak, leading to suboptimal payoffs. Experiments with Llama 3.1, Qwen3, and gpt‑oss confirm that acting optimally on decoded beliefs would improve outcomes in most games, highlighting a bottleneck in belief‑to‑action conversion.

By Jan Sobotka, Mustafa O. Karabag, Ufuk Topcu
llmssafety
More like this →
arXiv AI
Aug 28

Calibrated Enough to Know, Not Calibrated to Act: Fabricated Evidence Makes LLM Agents Commit to the Unknowable

The study shows that large language model (LLM) agents are far more likely to commit to a directional prediction when presented with a professional‑looking market panel than when asked the same question directly, with commitment rates rising from 6.5% to 54.0% across 12 frontier models. Even when the panel’s data is entirely fabricated, commitment still increases significantly, indicating that the authority of the presentation, rather than the truth of the information, drives confident action. The authors demonstrate that this act/don’t‑act decision gate is narrow, model‑specific, and can be mitigated through supervised fine‑tuning, though its effectiveness depends on response format and context. whyItMatters":"The findings reveal a specific vulnerability in LLMs where presentation style can override factual accuracy, highlighting the need for careful design and training to prevent misleading confidence in uncertain scenarios."

By Pranav Aggarwal
llmsagentsfine-tuning
More like this →
Hugging Face Trending Papers
Aug 27

Calibrated Enough to Know, Not Calibrated to Act: Fabricated Evidence Makes LLM Agents Commit to the Unknowable

The paper shows that large language model agents are far more likely to commit to a directional answer when presented with a professional-looking market panel, even if the panel’s data is fabricated. Across 12 frontier models, commitment rates jump from 6.5 % for a bare question to 54.0 % with evidence, and remain high (≈37 %) even when all numbers are invented. The study finds that the act/don’t‑act decision gate is the key failure point, can be trained to reduce false commitments, but is fragile to response format changes.

llmsagentsfine-tuning
More like this →