← Back to all news
arXiv Computation and Language By An-Sheng Jhang

Multi-Estimate Elicitation Does Not Uniformly Debias Language Model Judgments

Read the original on arXiv Computation and Language →

The study investigates whether prompting language models to provide five weighted estimates in a single response can reduce bias in their judgments. Across various bias tasks—anchoring, hindsight, base‑rate, conjunction, and status‑quo/default—the authors compare direct numerical answers with a multi‑estimate format, testing different source framings, random cues, and probability‑consistency constraints on three models (gpt‑6.1‑sol, claude‑sonnet‑5.5, and Qwen3‑8B). Results show that while direct elicitation reduces irrelevant‑anchor indices and status‑quo effects in two configurations, multi‑estimate elicitation does not consistently debias or attenuate responses, and the observed effects are limited by noise and lack of equivalence.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
Oct 8

The Long Road to the Same Answer: Cognitive Bias Under Escalating Reasoning Budgets in Large Language Models

The study investigates whether large language models (LLMs) that allocate extra computation during inference—termed reasoning models—reduce classic decision biases compared to their non‑reasoning counterparts. Using 30 vignettes covering six cognitive biases and varying token budgets up to 8,192 tokens, the authors find that reasoning models are not less biased, and increased deliberation does not reliably diminish bias magnitude. Only anchoring showed a bias in the human direction, while other biases either remained unchanged or moved further from human patterns, suggesting that test‑time reasoning does not guarantee rationality.

By Obada Kraishan
llmsbenchmarkssafety
More like this →
arXiv Machine Learning
1d ago

Same Answer, Different Future: Matched Predictions Do Not Imply Matched Belief Updating in Language Models

The paper investigates whether language models that produce the same probabilistic predictions after different types of evidence (textual or activation interventions) actually share the same underlying belief states. Using a Bernoulli inference task and an analytic Bayesian reference, the authors compare four textual routes and two activation-level interventions across three open-weight models, measuring both next-outcome continuations and explicit probabilistic estimates. They find that the format of evidence influences which readouts reflect its content, and that the effective evidence weight—how many new observations a belief is worth—varies widely across routes, with some elicitation questions altering inferred weights by orders of magnitude. In a one-shot steering scenario, an intervention that matches a natural record’s immediate prediction loses most of its effect when new evidence is introduced, demonstrating that agreement in current predictions does not guarantee equivalent belief updating.

By Weihan Li, Junhao Wu, Xinlei Chen, Tianshi Zheng
llms
More like this →